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The Institute of Education Sciences (lES) publishes practice guides in education 

to bring the best available evidence and expertise to bear on the types of systemic 
challenges that cannot currently be addressed by single interventions or programs. 
Authors of practice guides seldom conduct the types of systematic literature searches 
that are the backbone of a meta-analysis, although they take advantage of such work 
when it is already published. Instead, authors use their expertise to identify the 
most important research with respect to their recommendations, augmented by a 
search of recent publications to ensure that research citations are up-to-date. 

Unique to lES-sponsored practice guides is that they are subjected to rigorous exter- 
nal peer review through the same office that is responsible for independent review 
of other lES publications. A critical task for peer reviewers of a practice guide is to 
determine whether the evidence cited in support of particular recommendations is 
up-to-date and that studies of similar or better quality that point in a different di- 
rection have not been ignored. Because practice guides depend on the expertise of 
their authors and their group decisionmaking, the content of a practice guide is not 
and should not be viewed as a set of recommendations that in every case depends 
on and flows inevitably from scientific research. 

The goal of this practice guide is to formulate specific and coherent evidence-based 
recommendations for use by educators addressing the challenge of reducing the 
number of children who struggle with mathematics by using “response to interven- 
tion” (Rtl) as a means of both identifying students who need more help and provid- 
ing these students with high-quality interventions. The guide provides practical, 
clear information on critical topics related to Rtl and is based on the best available 
evidence as judged by the panel. Recommendations in this guide should not be 
construed to imply that no further research is warranted on the effectiveness of 
particular strategies used in Rtl for students struggling with mathematics. 
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Introduction 

Students struggling with mathematics may 
benefit from early interventions aimed at 
improving their mathematics ability and 
ultimately preventing subsequent failure. 
This guide provides eight specific recom- 
mendations intended to help teachers, 
principals, and school administrators use 
Response to Intervention (Rtl) to identify 
students who need assistance in mathe- 
matics and to address the needs of these 
students through focused interventions. 
The guide provides suggestions on how 
to carry out each recommendation and 
explains how educators can overcome 
potential roadblocks to implementing the 
recommendations. 

The recommendations were developed by 
a panel of researchers and practitioners 
with expertise in various dimensions of 
this topic. The panel includes a research 
mathematician active in issues related 
to K-8 mathematics education, two pro- 
fessors of mathematics education, sev- 
eral special educators, and a mathematics 
coach currently providing professional de- 
velopment in mathematics in schools. The 
panel members worked collaboratively to 
develop recommendations based on the 
best available research evidence and our 
expertise in mathematics, special educa- 
tion, research, and practice. 

The body of evidence we considered in de- 
veloping these recommendations included 
evaluations of mathematics interventions 
for low-performing students and students 
with learning disabilities. The panel con- 
sidered high-quality experimental and 
quasi-experimental studies, such as those 
meeting the criteria of the What Works 
Clearinghouse (http://www.whatworks. 
ed.gov), to provide the strongest evidence 
of effectiveness. We also examined stud- 
ies of the technical adequacy of batteries 
of screening and progress monitoring 
measures for recommendations relating 
to assessment. 



In some cases, recommendations reflect 
evidence-based practices that have been 
demonstrated as effective through rigor- 
ous research. In other cases, when such 
evidence is not available, the recommen- 
dations reflect what this panel believes are 
best practices. Throughout the guide, we 
clearly indicate the quality of the evidence 
that supports each recommendation. 

Each recommendation receives a rating 
based on the strength of the research evi- 
dence that has shown the effectiveness of a 
recommendation (table 1). These ratings — 
strong, moderate, or low — have been de- 
fined as follows: 

Strong refers to consistent and generaliz- 
able evidence that an intervention pro- 
gram causes better outcomes.' 

Moderate refers either to evidence from 
studies that allow strong causal conclu- 
sions but cannot be generalized with as- 
surance to the population on which a 
recommendation is focused (perhaps be- 
cause the findings have not been widely 
replicated) — or to evidence from stud- 
ies that are generalizable but have more 
causal ambiguity than offered by experi- 
mental designs (such as statistical models 
of correlational data or group comparison 
designs for which the equivalence of the 
groups at pretest is uncertain). 

Low refers to expert opinion based on rea- 
sonable extrapolations from research and 
theory on other topics and evidence from 
studies that do not meet the standards for 
moderate or strong evidence. 



1. Following WWC guidelines, we consider a posi- 
tive, statistically significant effect or large effect 
size (i.e., greater than 0.25) as an indicator of 
positive effects. 
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Table 1. institute of Education Sciences levels of evidence for practice guides 



Strong 


In general, characterization of the evidence for a recommendation as strong requires both 
studies with high internal validity (i.e., studies whose designs can support causal conclusions) 
and studies with high external validity (i.e., studies that in total include enough of the range 
of participants and settings on which the recommendation is focused to support the conclu- 
sion that the results can be generalized to those participants and settings). Strong evidence 
for this practice guide is operationalized as: 

• A systematic review of research that generally meets the standards of the What Works 
Clearinghouse (WWC) (see http://ies.ed.gov/ncee/wwc/) and supports the effectiveness of 
a program, practice, or approach with no contradictory evidence of similar quality; OR 

• Several well-designed, randomized controlled trials or well-designed quasi-experiments 
that generally meet the standards of WWC and support the effectiveness of a program, 
practice, or approach, with no contradictory evidence of similar quality; OR 

• One large, well-designed, randomized controlled, multisite trial that meets WWC standards 
and supports the effectiveness of a program, practice, or approach, with no contradictory 
evidence of similar quality; OR 

• Eor assessments, evidence of reliability and validity that meets the Standards for Educa- 
tional and Psychological Testing.** 


Moderate 


In general, characterization of the evidence for a recommendation as moderate requires stud- 
ies with high internal validity but moderate external validity, or studies with high external 
validity but moderate internal validity. In other words, moderate evidence is derived from 
studies that support strong causal conclusions but when generalization is uncertain, or stud- 
ies that support the generality of a relationship but when the causality is uncertain. Moderate 
evidence for this practice guide is operationalized as: 

• Experiments or quasi-experiments generally meeting the standards of WWC and sup- 
porting the effectiveness of a program, practice, or approach with small sample sizes 
and/or other conditions of implementation or analysis that limit generalizability and 
no contrary evidence; OR 

• Comparison group studies that do not demonstrate equivalence of groups at pre- 
test and therefore do not meet the standards of WWC but that (a) consistently show 
enhanced outcomes for participants experiencing a particular program, practice, or 
approach and (b) have no major flaws related to internal validity other than lack of 
demonstrated equivalence at pretest (e.g., only one teacher or one class per condition, 
unequal amounts of instructional time, highly biased outcome measures); OR 

• Correlational research with strong statistical controls for selection bias and for dis- 
cerning influence of endogenous factors and no contrary evidence; OR 

• for assessments, evidence of reliability that meets the Standards for Educational and 
Psychological Testing'* but with evidence of validity from samples not adequately rep- 
resentative of the population on which the recommendation is focused. 


Low 


In general, characterization of the evidence for a recommendation as low means that the 
recommendation is based on expert opinion derived from strong findings or theories in 
related areas and/or expert opinion buttressed by direct evidence that does not rise to 
the moderate or strong levels. Low evidence is operationalized as evidence not meeting 
the standards for the moderate or high levels. 



a. American Educational Research Association, American Psychological Association, and National Council on 
Measurement in Education (1999). 

b. Ibid. 
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The What Works Clearinghouse 
standards and their relevance to 
this guide 

The panel relied on WWC evidence stan- 
dards to assess the quality of evidence 
supporting mathematics intervention pro- 
grams and practices. The WWC addresses 
evidence for the causal validity of instruc- 
tional programs and practices according to 
WWC standards. Information about these 
standards is available at http://ies.ed.gov/ 
ncee/wwc/references/standards/. The 
technical quality of each study is rated and 
placed into one of three categories: 

• Meets Evidence Standards — for random- 
ized controlled trials and regression 
discontinuity studies that provide the 
strongest evidence of causal validity. 

• Meets Evidence Standards with Reser- 
vations — for all quasi-experimental 
studies with no design flaws and ran- 
domized controlled trials that have 
problems with randomization, attri- 
tion, or disruption. 

• Does Not Meet Evidence Screens — for 
studies that do not provide strong evi- 
dence of causal validity. 



Following the recommendations and sug- 
gestions for carrying out the recommen- 
dations, Appendix D presents information 
on the research evidence to support the 
recommendations. 

The panel would like to thank Kelly Hay- 
mond for her contributions to the analysis, 
the WWC reviewers for their contribution 
to the project, and Jo Ellen Kerr and Jamila 
Henderson for their support of the intricate 
logistics of the project. We also would like 
to thank Scott Cody for his oversight of the 
overall progress of the practice guide. 

Dr. Russell Gersten 
Dr. Sybilla Beckmann 
Dr. Benjamin Clarke 
Dr. Anne Foegen 
Ms. Laurel Marsh 
Dr. Jon R. Star 
Dr. Bradley Witzel 
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Struggling with 
Mathematics: Response 
to Intervention (Rtl) 
for Elementary and 
Middle Schools 

Overview 

Response to Intervention (Rtl) is an early de- 
tection, prevention, and support system that 
identifies struggling students and assists 
them before they fall behind. In the 2004 
reauthorization of the Individuals with Dis- 
abilities Education Act (PL 108-446), states 
were encouraged to use Rtl to accurately 
identify students with learning disabilities 
and encouraged to provide additional sup- 
ports for students with academic difficul- 
ties regardless of disability classification. 
Although many states have already begun to 
implement Rtl in the area of reading, Rtl ini- 
tiatives for mathematics are relatively new. 

Students’ low achievement in mathemat- 
ics is a matter of national concern. The re- 
cent National Mathematics Advisory Panel 
Report released in 2008 summarized the 
poor showing of students in the United 
States on international comparisons of 
mathematics performance such as the 
Trends in International Mathematics and 
Science Study (TIMSS) and the Program for 
International Student Assessment (PISA).^ 
A recent survey of algebra teachers as- 
sociated with the report identified key 
deficiencies of students entering algebra, 
including aspects of whole number arith- 
metic, fractions, ratios, and proportions.^ 
The National Mathematics Advisory Panel 



2. See, for example. National Mathematics Ad- 
visory Panel (2008) and Schmidt and Houang 
(2007). For more information on the TIMSS, see 
http://nces.ed.gov/timss/. For more information 
on PISA, see http://www.oecd.org. 

3. National Mathematics Advisory Panel (2008). 



concluded that all students should receive 
preparation from an early age to ensure 
their later success in algebra. In particular, 
the report emphasized the need for math- 
ematics interventions that mitigate and 
prevent mathematics difficulties. 

This panel believes that schools can use an 
Rtl framework to help struggling students 
prepare for later success in mathemat- 
ics. To date, little research has been con- 
ducted to identify the most effective ways 
to initiate and implement Rtl frameworks 
for mathematics. However, there is a rich 
body of research on effective mathematics 
interventions implemented outside an Rtl 
framework. Our goal in this practice guide 
is to provide suggestions for assessing 
students’ mathematics abilities and imple- 
menting mathematics interventions within 
an Rtl framework, in a way that reflects 
the best evidence on effective practices in 
mathematics interventions. 

Rtl begins with high-quality instruction 
and universal screening for all students. 
Whereas high-quality instruction seeks to 
prevent mathematics difficulties, screen- 
ing allows for early detection of difficul- 
ties if they emerge. Intensive interventions 
are then provided to support students 
in need of assistance with mathematics 
learning.^ Student responses to interven- 
tion are measured to determine whether 
they have made adequate progress and (1) 
no longer need intervention, (2) continue 
to need some intervention, or (3) need 
more intensive intervention. The levels of 
intervention are conventionally referred 
to as “tiers.” Rtl is typically thought of as 
having three tiers. ^ Within a three-tiered 
Rtl model, each tier is defined by specific 
characteristics. 



4. Fuchs, Fuchs, Craddock et al. (2008). 

5. Fuchs, Fuchs, and Vaughn (2008) make the 
case for a three-tier Rtl model. Note, however, 
that some states and school districts have imple- 
mented multitier intervention systems with more 
than three tiers. 
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• Tier 1 is the mathematics instruction 
that all students in a classroom receive. 
It entails universal screening of all stu- 
dents, regardless of mathematics profi- 
ciency, using valid measures to identify 
students at risk for future academic 
failure— so that they can receive early 
intervention.® There is no clear consen- 
sus on the characteristics of instruction 
other than that it is “high quality.”^ 

• In tier 2 interventions, schools provide 
additional assistance to students who 
demonstrate difficulties on screening 
measures or who demonstrate weak 
progress.® Tier 2 students receive sup- 
plemental small group mathematics 
instruction aimed at building targeted 
mathematics proficiencies.® These in- 
terventions are typically provided for 
20 to 40 minutes, four to five times each 
week.i® Student progress is monitored 
throughout the intervention. “ 

• Tier 3 interventions are provided to 
students who are not benefiting from 
tier 2 and require more intensive as- 
sistance. jier 3 usually entails one- 
on-one tutoring along with an appropri- 
ate mix of instructional interventions. 
In some cases, special education ser- 
vices are included in tier 3, and in oth- 
ers special education is considered an 
additional tier.^® Ongoing analysis of 



6. For reviews see Jiban and Deno (2007); Fuchs, 
Fuchs, Compton et al. (2007); Gersten, Jordan, 
and Flojo (2005). 

7. National Mathematics Advisory Panel (2008); 
National Research Council (2001). 

8. Fuchs, Fuchs, Craddock et al. (2008); Na- 
tional Joint Committee on Learning Disabilities 
(2005). 

9. Fuchs, Fuchs, Craddock et al. (2008). 

10. For example, see Jitendra et al. (1998) and 
Fuchs, Fuchs, Craddock et al. (2008). 

11. National Joint Committee on Learning Dis- 
abilities (2005). 

12. Fuchs, Fuchs, Craddock et al. (2008). 

13. Fuchs, Fuchs, Craddock et al. (2008); National 
Joint Committee on Learning Disabilities (2005). 



Student performance data is critical in 
this tier. Typically, specialized person- 
nel, such as special education teachers 
and school psychologists, are involved 
in tier 3 and special education services.^^ 
However, students often receive rele- 
vant mathematics interventions from a 
wide array of school personnel, includ- 
ing their classroom teacher. 

Summary of the Recommendations 

This practice guide offers eight recom- 
mendations for identifying and supporting 
students struggling in mathematics (table 
2). The recommendations are intended to 
be implemented within an Rtl framework 
(typically three-tiered). The panel chose to 
limit its discussion of tier 1 to universal 
screening practices (i.e., the guide does 
not make recommendations for general 
classroom mathematics instruction). Rec- 
ommendation 1 provides specific sugges- 
tions for conducting universal screening 
effectively. For Rtl tiers 2 and 3, recom- 
mendations 2 though 8 focus on the most 
effective content and pedagogical prac- 
tices that can be included in mathematics 
interventions. 

Throughout this guide, we use the term 
“interventionist” to refer to those teach- 
ing the intervention. At a given school, the 
interventionist may be the general class- 
room teacher, a mathematics coach, a spe- 
cial education instructor, other certified 
school personnel, or an instructional as- 
sistant. The panel recognizes that schools 
rely on different personnel to fill these 
roles depending on state policy, school 
resources, and preferences. 

Recommendation 1 addresses the type of 
screening measures that should be used in 
tier 1. We note that there is more research 
on valid screening measures for students in 



14. National Joint Committee on Learning Dis- 
abilities (2005). 
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Table 2. Recommendations and corresponding levels of evidence 



Recommendation 

Tier 1 


Level of evidence 


1. Screen all students to identify those at risk for potential mathematics 
difficulties and provide interventions to students identified as at risk. 

Tiers 2 and 3 


Moderate 


2. Instructional materials for students receiving interventions should 
focus intensely on in-depth treatment of whole numbers in kindergar- 
ten through grade 5 and on rational numbers in grades 4 through 8. 
These materials should be selected by committee. 


Low 


3. Instruction during the intervention should be explicit and systematic. 
This includes providing models of proficient problem solving, verbal- 
ization of thought processes, guided practice, corrective feedback, and 
frequent cumulative review. 


Strong 


4. Interventions should include instruction on solving word problems 
that is based on common underlying structures. 


Strong 


5. Intervention materials should include opportunities for students to 
work with visual representations of mathematical ideas and interven- 
tionists should be proficient in the use of visual representations of 
mathematical ideas. 


Moderate 


6. Interventions at all grade levels should devote about 10 minutes in each 
session to building fluent retrieval of basic arithmetic facts. 


Moderate 


7. Monitor the progress of students receiving supplemental instruction 
and other students who are at risk. 


Low 


8. Include motivational strategies in tier 2 and tier 3 interventions. 


Low 


Source; Authors’ compilation based on analysis described in text. 
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kindergarten through grade 2/^ but there 
are also reasonable strategies to use for stu- 
dents in more advanced grades. We stress 
that no one screening measure is perfect 
and that schools need to monitor the prog- 
ress of students who score slightly above or 
slightly below any screening cutoff score. 

Recommendations 2 though 6 address the 
content of tier 2 and tier 3 interventions 
and the types of instructional strategies 
that should be used. In recommendation 2, 
we translate the guidance by the National 
Mathematics Advisory Panel (2008) and 
the National Council of Teachers of Math- 
ematics Curriculum Focal Points (2006) 
into suggestions for the content of inter- 
vention curricula. We argue that the math- 
ematical focus and the in-depth coverage 
advocated for proficient students are also 
necessary for students with mathematics 
difficulties. For most students, the content 
of interventions will include foundational 
concepts and skills introduced earlier in 
the student’s career but not fully under- 
stood and mastered. Whenever possible, 
links should be made between founda- 
tional mathematical concepts in the inter- 
vention and grade-level material. 

At the center of the intervention recom- 
mendations is that instruction should be 
systematic and explicit (recommendation 
3). This is a recurrent theme in the body 
of valid scientific research. We explore 
the multiple meanings of explicit instruc- 
tion and indicate which components of 
explicit instruction appear to be most re- 
lated to improved student outcomes. We 
believe this information is important for 
districts and state departments to have 
as they consider selecting materials and 



15. Gersten, Jordan, and Flojo (2005); Gersten, 
Clarke, and Jordan (2007). 

16. Jiban and Deno (2007); Foegen, Jiban, and 
Deno (2007). 

17. Darch, Gamine, and Gersten (1984); Fuchs 
et al. (2003a); Jitendra et al. (1998); Schunk and 
Cox (1986); Tournaki (2003); Wilson and Sindelar 
(1991). 



providing professional development for 
interventionists. 

Next, we highlight several areas of re- 
search that have produced promising find- 
ings in mathematics interventions. These 
include systematically teaching students 
about the problem types associated with 
a given operation and its inverse (such as 
problem types that indicate addition and 
subtraction) (recommendation 4).^® We also 
recommend practices to help students 
translate abstract symbols and numbers 
into meaningful visual representations 
(recommendation 5).^^ Another feature 
that we identify as crucial for long-term 
success is systematic instruction to build 
quick retrieval of basic arithmetic facts 
(recommendation 6). Some evidence exists 
supporting the allocation of time in the in- 
tervention to practice fact retrieval using 
flash cards or computer software. There 
is also evidence that systematic work with 
properties of operations and counting 
strategies (for younger students) is likely 
to promote growth in other areas of math- 
ematics beyond fact retrieval. 

The final two recommendations address 
other considerations in implementing tier 
2 and tier 3 interventions. Recommenda- 
tion 7 addresses the importance of moni- 
toring the progress of students receiving 



18. Jitendra et al. (1998); Xin, Jitendra, and Deat- 
line-Buchman (2005); Darch, Gamine, and Gersten 
(1984); Fuchs et al. (2003a); Fuchs et al. (2003b); 
Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, 
and Finelli (2004); Fuchs, Fuchs, Graddock et al. 
(2008) Fuchs, Seethaler et al. (2008). 

19. Artus and Dyrek (1989); Butler et al. (2003); 
Darch, Gamine, and Gersten (1984); Fuchs et 
al. (2005); Fuchs, Seethaler et al. (2008); Fuchs, 
Powell et al. (2008); Fuchs, Fuchs, Graddock et 
al. (2008); Jitendra et al. (1998); Walker and Po- 
teet (1989); Wilson and Sindelar (1991); Witzel 
(2005); Witzel, Mercer, and Miller (2003); Wood- 
ward (2006). 

20. Bernie-Smith (1991); Fuchs, Seethaler et al. 
(2008); Fuchs et al. (2005); Fuchs, Fuchs, Hamlett 
et al. (2006); Fuchs, Powell et al. (2008). 

21. Tournaki (2003); Woodward (2006). 
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interventions. Specific types of formative 
assessment approaches and measures are 
described. We argue for two types of ongo- 
ing assessment. One is the use of curricu- 
lum-embedded assessments that gauge how 
well students have learned the material in 
that day’s or week’s lesson(s). The panel 
believes this information is critical for in- 
terventionists to determine whether they 
need to spend additional time on a topic. It 
also provides the interventionist and other 
school personnel with information that 
can be used to place students in groups 
within tiers. In addition, we recommend 
that schools regularly monitor the prog- 
ress of students receiving interventions 



and those with scores slightly above or 
below the cutoff score on screening mea- 
sures with broader measures of mathemat- 
ics proficiency. This information provides 
the school with a sense of how the overall 
mathematics program (including tier 1, tier 
2, and tier 3) is affecting a given student. 

Recommendation 8 addresses the impor- 
tant issue of motivation. Because many of 
the students struggling with mathematics 
have experienced failure and frustration 
by the time they receive an intervention, 
we suggest tools that can encourage active 
engagement of students and acknowledge 
student accomplishments. 



(8) 



Scope of the 
practice guide 

Our goal is to provide evidence-based sug- 
gestions for screening students for mathe- 
matics difficulties, providing interventions 
to students who are struggling, and moni- 
toring student responses to the interven- 
tions. Rtl intentionally cuts across the bor- 
ders of special and general education and 
involves school-wide collaboration. There- 
fore, our target audience for this guide in- 
cludes teachers, special educators, school 
psychologists and counselors, as well as 
administrators. Descriptions of the ma- 
terials and instructional content in tier 2 
and tier 3 interventions may be especially 
useful to school administrators selecting 
interventions, while recommendations 
that relate to content and pedagogy will 
be most useful to interventionists.^^ 

The focus of this guide is on providing 
Rtl interventions in mathematics for stu- 
dents in kindergarten through grade 8. This 
broad grade range is in part a response 
to the recent report of the National Math- 
ematics Advisory Panel (2008), which em- 
phasized a unified progressive approach 
to promoting mathematics proficiency for 
elementary and middle schools. Moreover, 
given the growing number of initiatives 
aimed at supporting students to succeed 
in algebra, the panel believes it essential 
to provide tier 2 and tier 3 interventions to 
struggling students in grades 4 through 8. 
Because the bulk of research on mathemat- 
ics interventions has focused on students 
in kindergarten through grade 4, some rec- 
ommendations for students in older grades 
are extrapolated from this research. 



22. Interventionists may be any number of school 
personnel, including classroom teachers, special 
educators, school psychologists, paraprofession- 
als, and mathematics coaches and specialists. 
The panel does not specify the interventionist. 



The scope of this guide does not include 
recommendations for special education 
referrals. Although enhancing the valid- 
ity of special education referrals remains 
important and an issue of ongoing discus- 
sion23 and research, yye do not address 
it in this practice guide, in part because 
empirical evidence is lacking. 

The discussion of tier 1 in this guide re- 
volves only around effective screening, be- 
cause recommendations for general class- 
room mathematics instruction were beyond 
the scope of this guide. For this reason, 
studies of effective general mathematics 
instruction practices were not included in 
the evidence base for this guide. 

The studies reviewed for this guide in- 
cluded two types of comparisons among 
groups. First, several studies of tier 2 in- 
terventions compare students receiving 
multicomponent tier 2 interventions with 
students receiving only routine classroom 
instruction.^® This type of study provides 
evidence of the effectiveness of providing 
tier 2 interventions but does not permit 
conclusions about which component is 
most effective. The reason is that it is not 
possible to identify whether one particular 
component or a combination of compo- 
nents within a multicomponent interven- 
tion produced an effect. Second, several 



23. Kavale and Spaulding (2008); Fuchs, Fuchs, 
and Vaughn (2008); VanDerHeyden, Witt, and 
Gilbertson (2007). 

24. Fuchs, Fuchs, Compton et al. (2006). 

25. There were a few exceptions in which general 
mathematics instruction studies were included in 
the evidence base. When the effects of a general 
mathematics instruction program were specified 
for low-achieving or disabled students and the 
intervention itself appeared applicable to teach- 
ing tier 2 or tier 3 (e.g., teaching a specific opera- 
tional strategy), we included them in this study. 
Note that disabled students were predominantly 
learning disabled. 

26. For example, Fuchs, Seethaler et al. (2008) 
examined the effects of providing supplemen- 
tal tutoring (i.e., a tier 2 intervention) relative to 
regular classroom instruction (i.e., tier 1). 
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SCOPE OF THE PRACTICE GUIDE 



Other studies examined the effects of two 
methods of tier 2 or tier 3 instruction.^^ 
This type of study offers evidence for the 
effectiveness of one approach to teaching 
within a tier relative to another approach 
and assists with identifying the most ben- 
eficial approaches for this population. 

The panel reviewed only studies for prac- 
tices that sought to improve student math- 
ematics outcomes. The panel did not con- 
sider interventions that improved other 
academic or behavioral outcomes. Instead, 
the panel focused on practices that ad- 
dressed the following areas of mathematics 
proficiency; operations {either computation 



27. For example, Tournaki (2003) examined the 
effects of providing supplemental tutoring in an 
operations strategy (a tier 2 intervention) relative 
to supplemental tutoring with a drill and practice 
approach (also a tier 2 intervention). 



or estimation), concepts (knowledge of 
properties of operations, concepts involv- 
ing rational numbers, prealgebra con- 
cepts), problem solving (word problems), 
and measures of general mathematics 
achievement. Measures of fact fluency 
were also included because quick retrieval 
of basic arithmetic facts is essential for 
success in mathematics and a persistent 
problem for students with difficulties in 
mathematics.^^ 

Technical terms related to mathematics 
and technical aspects of assessments (psy- 
chometrics) are defined in a glossary at the 
end of the recommendations. 



28. Geary (2004); Jordan, Hanich, and Kaplan 
(2003). 
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Checklist for carrying out the 
recommendations 

Recommendation 1. Screen all 
students to identify those at risk for 
potential mathematics difficulties and 
provide interventions to students 
identified as at risk. 

□ As a district or school sets up a screen- 
ing system, have a team evaluate potential 
screening measures. The team should se- 
lect measures that are efficient and reason- 
ably reliable and that demonstrate predic- 
tive validity. Screening should occur in the 
beginning and middle of the year. 

CH Select screening measures based on 
the content they cover, with an emphasis 
on critical instructional objectives for each 
grade. 

□ In grades 4 through 8, use screen- 
ing data in combination with state testing 
results. 

CH Use the same screening tool across a 
district to enable analyzing results across 
schools. 

Recommendation 2. Instructional 
materials for students receiving 
interventions should focus intensely 
on in-depth treatment of whole 
numbers in kindergarten through 
grade 5 and on rational numbers in 
grades 4 through 8. These materials 
should be selected by committee. 

□ For students in kindergarten through 
grade 5, tier 2 and tier 3 interventions 
should focus almost exclusively on prop- 
erties of whole numbers and operations. 
Some older students struggling with 
whole numbers and operations would 
also benefit from in-depth coverage of 
these topics. 

□ Fortier 2 and tier 3 students in grades 
4 through 8, interventions should focus on 



in-depth coverage of rational numbers as 
well as advanced topics in whole number 
arithmetic (such as long division). 

□ Districts should appoint committees, 
including experts in mathematics instruc- 
tion and mathematicians with knowledge 
of elementary and middle school math- 
ematics curricula, to ensure that specific 
criteria are covered in-depth in the cur- 
riculum they adopt. 

Recommendation 3. Instruction during 
the intervention should be explicit and 
systematic. This includes providing 
models of proficient problem solving, 
verbalization of thought processes, 
guided practice, corrective feedback, 
and frequent cumulative review. 

□ Ensure that instructional materials are 
systematic and explicit. In particular, they 
should include numerous clear models of 
easy and difficult problems, with accom- 
panying teacher think-alouds. 

CH Provide students with opportunities 
to solve problems in a group and commu- 
nicate problem-solving strategies. 

□ Ensure that instructional materials in- 
clude cumulative review in each session. 

Recommendation 4. Interventions 
should include instruction on solving 
word problems that is based on 
common underlying structures. 

CH Teach students about the structure of 
various problem types, how to categorize 
problems based on structure, and howto 
determine appropriate solutions for each 
problem type. 

CH Teach students to recognize the com- 
mon underlying structure between famil- 
iar and unfamiliar problems and to transfer 
known solution methods from familiar to 
unfamiliar problems. 
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CHECKLIST FOR CARRYING OUT THE RECOMMENDATIONS 



Recommendation 5. Intervention 
materials should include opportunities 
for students to work with visual 
representations of mathematical 
ideas and interventionists should 
be proficient in the use of visual 
representations of mathematical ideas. 

□ Use visual representations such as 
number lines, arrays, and strip diagrams. 

CH If visuals are not sufficient for develop- 
ing accurate abstract thought and answers, 
use concrete manipulatives first. Although 
this can also be done with students in upper 
elementary and middle school grades, use 
of manipulatives with older students should 
be expeditious because the goal is to move 
toward understanding of — and facility 
with — visual representations, and finally, to 
the abstract. 

Recommendation 6. Interventions at 
all grade levels should devote about 
10 minutes in each session to building 
fluent retrieval of basic arithmetic facts. 

CH Provide about 10 minutes per ses- 
sion of instruction to build quick retrieval 
of basic arithmetic facts. Consider using 
technology, flash cards, and other materi- 
als for extensive practice to facilitate au- 
tomatic retrieval. 

□ For students in kindergarten through 
grade 2, explicitly teach strategies for ef- 
ficient counting to improve the retrieval of 
mathematics facts. 



CH Teach students in grades 2 through 
8 how to use their knowledge of proper- 
ties, such as commutative, associative, 
and distributive law, to derive facts in 
their heads. 

Recommendation 7. Monitor the 
progress of students receiving 
supplemental instruction and other 
students who are at risk. 

□ Monitor the progress of tier 2, tier 3, 
and borderline tier 1 students at least once 
a month using grade-appropriate general 
outcome measures. 

□ Use curriculum-embedded assess- 
ments in interventions to determine 
whether students are learning from the 
intervention. These measures can be used 
as often as every day or as infrequently as 
once every other week. 

□ Use progress monitoring data to re- 
group students when necessary. 

Recommendation 8. Include 
motivational strategies in tier 2 and 
tier 3 interventions. 

CH Reinforce or praise students for their 
effort and for attending to and being en- 
gaged in the lesson. 

CH Consider rewarding student accom- 
plishments. 

CH Allow students to chart their progress 
and to set goals for improvement. 
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Recommendation 1. 
Screen all students to 
identify those at risk for 
potential mathematics 
difficulties and provide 
interventions to 
students identified 
as at risk. 

The panel recommends that schools 
and districts systematically use 
universal screening to screen all 
students to determine which students 
have mathematics difficulties and 
require research-based interventions. 
Schools should evaluate and select 
screening measures based on their 
reliability and predictive validity, with 
particular emphasis on the measures’ 
specificity and sensitivity. Schools 
should also consider the efficiency of 
the measure to enable screening many 
students in a short time. 

Level of evidence: Moderate 

The panel judged the level of evidence sup- 
porting this recommendation to be mod- 
erate. This recommendation is based on a 
series of high-quality correlational studies 
with replicated findings that show the abil- 
ity of measures to predict performance in 
mathematics one year after administration 
(and in some cases two years).^*^ 

Brief summary of evidence to 
support the recommendation 

A growing body of evidence suggests that 
there are several valid and reliable ap- 
proaches for screening students in the pri- 
mary grades. All these approaches target 



29. For reviews see Jiban and Deno (2007); Fuchs, 
Fuchs, Compton et al. (2007); Gersten, Jordan, 
and Flojo (2005). 



aspects of what is often referred to as 
number sense. They assess various as- 
pects of knowledge of whole numbers — 
properties, basic arithmetic operations, 
understanding of magnitude, and applying 
mathematical knowledge to word prob- 
lems. Some measures contain only one 
aspect of number sense (such as magni- 
tude comparison) and others assess four 
to eight aspects of number sense. The sin- 
gle-component approaches with the best 
ability to predict students’ subsequent 
mathematics performance include screen- 
ing measures of students’ knowledge of 
magnitude comparison and/or strategic 
counting. The broader, multicomponent 
measures seem to predict with slightly 
greater accuracy than single-component 
measures. 

Effective approaches to screening vary in 
efficiency, with some taking as little as 5 
minutes to administer and others as long 
as 20 minutes. Multicomponent measures, 
which by their nature take longer to ad- 
minister, tend to be time-consuming for 
administering to an entire school popu- 
lation. Timed screening measures^^ and 
untimed screening measures^^ have been 
shown to be valid and reliable. 

For the upper elementary grades and mid- 
dle school, we were able to locate fewer 
studies. They suggest that brief early 
screening measures that take about 10 
minutes and cover a proportional sam- 
pling of grade-level objectives are reason- 
able and provide sufficient evidence of reli- 
ability.^^ At the current time, this research 
area is underdeveloped. 



30. Berch (2005); Dehaene (1999); Okamoto and 
Case (1996); Gersten and Chard (1999). 

31. Gersten, Jordan, and Flojo (2005). 

32. Fuchs, Fuchs, Compton et al. (2007). 

33. For example, Clarke and Shinn (2004). 

34. For example, Okamoto and Case (1996). 

35. Jiban and Deno (2007); Foegen, Jiban, and 
Deno (2007). 
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RECOMMENDATION 1. SCREEN ALL STUDENTS TO IDENTIFY THOSE AT RISK 



How to carry out this 
recommendation 



1. As a district or school sets up a screen- 
ing system, have a team evaluate potential 
screening measures. The team should select 
measures that are efficient and reasonably 
reliable and that demonstrate predictive va- 
lidity. Screening should occur in the begin- 
ning and middle of the year. 

The team that selects the measures should 
include individuals with expertise in mea- 
surement (such as a school psychologist or 
a member of the district research and eval- 
uation division) and those with expertise in 
mathematics instruction. In the opinion of 
the panel, districts should evaluate screen- 
ing measures on three dimensions. 

• Predictive validity is an index of how 
well a score on a screening measure 
earlier in the year predicts a student’s 
later mathematics achievement. Greater 
predictive validity means that schools 
can be more confident that decisions 
based on screening data are accurate. 
In general, we recommend that schools 
and districts employ measures with 
predictive validity coefficients of at 
least .60 within a school year.^® 

• Reliability is an index of the consistency 
and precision of a measure. We recom- 
mend measures with reliability coeffi- 
cients of .80 or higher.^^ 

• Efficiency is how quickly the universal 
screening measure can be adminis- 
tered, scored, and analyzed for all the 
students. As a general rule, we suggest 
that a screening measure require no 



36. A coefficient of .0 inciicates that there is no 
relation between the early and later scores, and 
a coefficient of 1.0 indicates a perfect positive 
relation between the scores. 

37. A coefficient of .0 indicates that there is no 
relation between the two scores, and a coeffi- 
cient of 1.0 indicates a perfect positive relation 
between the scores. 



more than 20 minutes to administer, 
which enables collecting a substantial 
amount of information in a reasonable 
time frame. Note that many screening 
measures take five minutes or less.^* We 
recommend that schools select screen- 
ing measures that have greater effi- 
ciency if their technical adequacy (pre- 
dictive validity, reliability, sensitivity, 
and specificity) is roughly equivalent 
to less efficient measures. Remember 
that screening measures are intended 
for administration to all students in a 
school, and it may be better to invest 
more time in diagnostic assessment of 
students who perform poorly on the 
universal screening measure. 

Keep in mind that screening is just a means 
of determining which students are likely to 
need help. If a student scores poorly on a 
screening measure or screening battery — 
especially if the score is at or near a cut 
point, the panel recommends monitoring 
her or his progress carefully to discern 
whether extra instruction is necessary. 

Developers of screening systems recom- 
mend that screening occur at least twice 
a year (e.g., fall, winter, and/or spring).^^ 
This panel recommends that schools alle- 
viate concern about students just above or 
below the cut score by screening students 
twice during the year. The second screen- 
ing in the middle of the year allows another 
check on these students and also serves to 
identify any students who may have been at 
risk and grown substantially in their mathe- 
matics achievement— or those who were on- 
track at the beginning of the year but have 
not shown sufficient growth. The panel 
considers these two universal screenings 
to determine student proficiency as distinct 
from progress monitoring (Recommenda- 
tion 7), which occurs on a more frequent 



38. Foegen, Jiban, and Deno (2007); Fuchs, Fuchs, 
Compton et al. (2007); Gersten, Clarke, and Jordan 
(2007). 

39. Kaminski et al. (2008); Shinn (1989). 



( 14 ) 



RECOMMENDATION 1 . SCREEN ALL STUDENTS TO IDENTIFY THOSE AT RISK 



basis (e.g., weekly or monthly) with a select 
group of intervention students in order to 
monitor response to intervention. 



2. Select screening measures based on the 
content they cover, with an emphasis on crit- 
ical instructional objectives for each grade. 

The panel believes that content covered 
in a screening measure should reflect the 
instructional objectives for a student’s 
grade level, with an emphasis on the most 
critical content for the grade level. The Na- 
tional Council of Teachers of Mathematics 
(2006) released a set of focal points for 
each grade level designed to focus instruc- 
tion on critical concepts for students to 
master within a specific grade. Similarly, 
the National Mathematics Advisory Panel 
(2008) detailed a route to preparing all 
students to be successful in algebra. In the 
lower elementary grades, the core focus of 
instruction is on building student under- 
standing of whole numbers. As students 
establish an understanding of whole num- 
bers, rational numbers become the focus 
of instruction in the upper elementary 
grades. Accordingly, screening measures 
used in the lower and upper elementary 
grades should have items designed to as- 
sess student’s understanding of whole and 
rational number concepts — as well as com- 
putational proficiency. 



3. In grades 4 through 8, use screening data 
in combination with state testing results. 

In the panel’s opinion, one viable option 
that schools and districts can pursue is to 
use results from the previous year’s state 
testing as a first stage of screening. Students 
who score below or only slightly above a 
benchmark would be considered for sub- 
sequent screening and/or diagnostic or 
placement testing. The use of state testing 
results would allow districts and schools 
to combine a broader measure that covers 
more content with a screening measure that 
is narrower but more focused. Because of 
the lack of available screening measures at 



these grade levels, districts, county offices, 
or state departments may need to develop 
additional screening and diagnostic mea- 
sures or rely on placement tests provided 
by developers of intervention curricula. 



4. Use the same screening tool across a district 
to enable analyzing results across schools. 

The panel recommends that all schools 
within a district use the same screening 
measure and procedures to ensure ob- 
jective comparisons across schools and 
within a district. Districts can use results 
from screening to inform instructional de- 
cisions at the district level. For example, 
one school in a district may consistently 
have more students identified as at risk, 
and the district could provide extra re- 
sources or professional development to 
that school. The panel recommends that 
districts use their research and evaluation 
staff to reevaluate screening measures an- 
nually or biannually. This entails exam- 
ining how screening scores predict state 
testing results and considering resetting 
cut scores or other data points linked to 
instructional decisionmaking. 

Potential roadblocks and solutions 

Roadblock 1.1. Districts and school person- 
nel may face resistance in allocating time re- 
sources to the collection of screening data. 

Suggested Approach. The issue of time 
and personnel is likely to be the most sig- 
nificant obstacle that districts and schools 
must overcome to collect screening data. 
Collecting data on all students will require 
structuring the data collection process to 
be efficient and streamlined. 

The panel notes that a common pitfall is 
a long, drawn-out data collection process, 
with teachers collecting data in their class- 
rooms “when time permits.” If schools are 
allocating resources (such as providing an 
intervention to students with the 20 low- 
est scores in grade 1), they must wait until 
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RECOMMENDATION 1. SCREEN ALL STUDENTS TO IDENTIFY THOSE AT RISK 



all the data have been collected across 
classrooms, thus delaying the delivery 
of needed services to students. Further- 
more, because many screening measures 
are sensitive to instruction, a wide gap 
between when one class is assessed and 
another is assessed means that many stu- 
dents in the second class will have higher 
scores than those in the first because they 
were assessed later. 

One way to avoid these pitfalls is to use data 
collection teams to screen students in a 
short period of time. The teams can consist 
of teachers, special education staff includ- 
ing such specialists as school psychologists. 
Title 1 staff, principals, trained instructional 
assistants, trained older students, and/or 
local college students studying child devel- 
opment or school psychology. 

Roadblock 1.2. Implementing universal 
screening is likely to raise questions such 
as, “Why are we testing students who are 
doing fine?” 

Suggested Approach. Collecting data 
on all students is new for many districts 
and schools (this may not be the case for 
elementary schools, many of which use 
screening assessments in reading).^° But 
screening allows schools to ensure that all 
students who are on track stay on track 
and collective screening allows schools to 
evaluate the impact of their instruction 
on groups of students (such as all grade 
2 students). When schools screen all stu- 
dents, a distribution of achievement from 
high to low is created. If students consid- 
ered not at risk were not screened, the 
distribution of screened students would 
consist only of at-risk students. This could 
create a situation where some students at 
the “top” of the distribution are in real- 
ity at risk but not identified as such. For 
upper-grade students whose scores were 



40. U.S. Department of Education, Office of Plan- 
ning, Evaluation and Policy Development, Policy 
and Program Studies Service (2006). 



high on the previous spring’s state as- 
sessment, additional screening typically 
is not required. 

Roadblock 1.3. Screening measures may 
identify students who do not need services 
and not identify students who do need 
services. 

Suggested Approach. All screening mea- 
sures will misidentify some students as 
either needing assistance when they do 
not (false positive) or not needing assis- 
tance when they do (false negative). When 
screening students, educators will want to 
maximize both the number of students 
correctly identified as at risk— a measure’s 
sensitivity — and the number of students 
correctly identified as not at risk— a mea- 
sure’s specificity. As illustrated in table 3, 
screening students to determine risk can 
result in four possible categories indicated 
by the letters A, B, C, and D. Using these 
categories, sensitivity is equal to A/(A-e C) 
and specificity is equal to D/(B -e D). 



Table 3. Sensitivity and specificity 







STUDENTS 
ACTUALLY AT RISK 






Yes 


No 


STUDENTS 

IDENTIFIED 


Yes 


A (true 
positives) 


B (false 
positives) 


AS BEING 
AT RISK 


No 


C (false 
negatives) 


D (true 
negatives) 



The sensitivity and specificity of a mea- 
sure depend on the cut score to classify 
children at risk.'^^ If a cut score is high 
(where all students below the cut score are 
considered at risk), the measure will have 
a high degree of sensitivity because most 
students who truly need assistance will be 



41. Sensitivity and specificity are also influenced 
by the discriminant validity of the measure and 
its individual items. Measures with strong item 
discrimination are more likely to correctly iden- 
tify students’ risk status. 
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identified as at risk. But the measure will 
have low specificity since many students 
who do not need assistance will also be 
identified as at risk. Similarly, if a cut score 
is low, the sensitivity will be lower (some 
students in need of assistance may not be 
identified as at risk), whereas the specific- 
ity will be higher (most students who do 
not need assistance will not be identified 
as at risk). 

Schools need to be aware of this tradeoff 
between sensitivity and specificity, and 
the team selecting measures should be 
aware that decisions on cut scores can be 
somewhat arbitrary. Schools that set a cut 
score too high run the risk of spending re- 
sources on students who do not need help, 
and schools that set a cut score too low run 
the risk of not providing interventions to 
students who are at risk and need extra in- 
struction. If a school or district consistently 
finds that students receiving intervention 
do not need it, the measurement team 
should consider lowering the cut score. 

Roadblock 1.4. Screening data may iden- 
tify large numbers of students who are at 
risk and schools may not immediately have 
the resources to support all at-risk students. 
This will be a particularly severe problem 
in low-performing Title I schools. 

Suggested Approach. Districts and 
schools need to consider the amount of 
resources available and the allocation of 



those resources when using screening 
data to make instructional decisions. Dis- 
tricts may find that on a nationally normed 
screening measure, a large percentage of 
their students (such as 60 percent) will be 
classified as at risk. Districts will have to 
determine the resources they have to pro- 
vide interventions and the number of stu- 
dents they can serve with their resources. 
This may mean not providing interven- 
tions at certain grade levels or providing 
interventions only to students with the 
lowest scores, at least in the first year of 
implementation. 

There may also be cases when schools 
identify large numbers of students at risk 
in a particular area and decide to pro- 
vide instruction to all students. One par- 
ticularly salient example is in the area of 
fractions. Multiple national assessments 
show many students lack proficiency in 
fractions,'’^ so a school may decide that, 
rather than deliver interventions at the 
individual child level, they will provide a 
school-wide intervention to all students. A 
school-wide intervention can range from a 
supplemental fractions program to profes- 
sional development involving fractions. 



42. National Mathematics Advisory Panel 
(2008); Lee, Grigg, and Dion (2007). 
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Recommendation 2. 
Instructional materials 
for students receiving 
interventions should 
focus intensely on 
in-depth treatment 
of whole numbers in 
kindergarten through 
grade 5 and on rational 
numbers in grades 
4 through 8. These 
materials should be 
selected by committee. 

The panel recommends that individuals 
knowledgeable in instruction and 
mathematics look for interventions that 
focus on whole numbers extensively 
in kindergarten through grade 5 and 
on rational numbers extensively in 
grades 4 through 8. In all cases, the 
specific content of the interventions will 
be centered on building the student’s 
foundational proficiencies. In making 
this recommendation, the panel is 
drawing on consensus documents 
developed by experts from mathematics 
education and research mathematicians 
that emphasized the importance of 
these topics for students in general.'^^ 

We conclude that the coverage of 
fewer topics in more depth, and 
with coherence, is as important, and 
probably more important, for students 
who struggle with mathematics. 



43. National Council of Teachers of Mathemat- 
ics (2006); National Mathematics Advisory Panel 
(2008). 



Level of evidence: Low 

The panel judged the level of evidence 
supporting this recommendation to be low. 
This recommendation is based on the pro- 
fessional opinion of the panel and several 
recent consensus documents that reflect 
input from mathematics educators and re- 
search mathematicians involved in issues 
related to kindergarten through grade 12 
mathematics education.^^ 

Brief summary of evidence to 
support the recommendation 

The documents reviewed demonstrate a 
growing professional consensus that cov- 
erage of fewer mathematics topics in more 
depth and with coherence is important 
for all students.'*^ Milgram and Wu (2005) 
suggested that an intervention curriculum 
for at-risk students should not be over- 
simplified and that in-depth coverage of 
key topics and concepts involving whole 
numbers and then rational numbers is 
critical for future success in mathematics. 
The National Council of Teachers of Math- 
ematics (NCTM) Curriculum Focal Points 
(2006) called for the end of brief ventures 
into many topics in the course of a school 
year and also suggested heavy emphasis on 
instruction in whole numbers and rational 
numbers. This position was reinforced by 
the 2008 report of the National Mathematics 
Advisory Panel (NMAP), which provided de- 
tailed benchmarks and again emphasized 
in-depth coverage of key topics involving 
whole numbers and rational numbers as 
crucial for all students. Although the latter 
two documents addressed the needs of all 
students, the panel concludes that the in- 
depth coverage of key topics is especially 



44. National Council of Teachers of Mathemat- 
ics (2006); National Mathematics Advisory Panel 
(2008); Milgram and Wu (2005). 

45. National Mathematics Advisory Panel (2008); 
Schmidt and Houang (2007); Milgram and Wu 
(2005); National Council of Teachers of Math- 
ematics (2006). 
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RECOMMENDATION 2. INSTRUCTIONAL MATERIALS FOR STUDENTS RECEIVING INTERVENTIONS 



important for students who struggle with 
mathematics. 

How to carry out this 
recommendation 



1. For students in kindergarten through 
grade 5, tier 2 and tier 3 interventions should 
focus almost exclusively on properties of 
whole numbers'^® and operations. Some 
older students struggling with whole num- 
bers and operations would also benefit from 
in-depth coverage of these topics. 

In the panel’s opinion, districts should 
review the interventions they are con- 
sidering to ensure that they cover whole 
numbers in depth. The goal is proficiency 
and mastery, so in-depth coverage with 
extensive review is essential and has 
been articulated in the NCTM Curriculum 
Focal Points (2006) and the benchmarks 
determined by the National Mathematics 
Advisory Panel (2008). Readers are recom- 
mended to review these documents.'*’ 

Specific choices for the content of interven- 
tions will depend on the grade level and 
proficiency of the student, but the focus 
for struggling students should be on whole 
numbers. For example, in kindergarten 
through grade 2, intervention materials 
would typically include significant atten- 
tion to counting (e.g., counting up), num- 
ber composition, and number decomposi- 
tion (to understand place-value multidigit 
operations). Interventions should cover the 
meaning of addition and subtraction and 



46. Properties of numbers, including the associa- 
tive, commutative, and distributive properties. 

47. More information on the National Mathemat- 
ics Advisory Panel (2008) report is available at 
www.ed.gov/about/bdscomm/list/mathpanel/ 
index.html. More information on the National 
Council of Teachers of Mathematics Curricu- 
lum Focal Points is available at www.nctm.org/ 
focalpoints. Documents elaborating the National 
Council of Teachers of Mathematics Curriculum 
Focal Points are also available (see Beckmann et 
al., 2009). For a discussion of why this content is 
most relevant, see Milgram and Wu (2005). 



the reasoning that underlies algorithms for 
addition and subtraction of whole num- 
bers, as well as solving problems involv- 
ing whole numbers. This focus should in- 
clude understanding of the base-10 system 
(place value). 

Interventions should also include materi- 
als to build fluent retrieval of basic arith- 
metic facts (see recommendation 6). Ma- 
terials should extensively use — and ask 
students to use— visual representations of 
whole numbers, including both concrete 
and visual base-10 representations, as well 
as number paths and number lines (more 
information on visual representations is 
in recommendation 5). 



2. For tier 2 and tier 3 students in grades 4 
through 8, interventions should focus on in- 
depth coverage of rational numbers as well 
as advanced topics in whole number arith- 
metic (such as long division). 

The panel believes that districts should 
review the interventions they are consid- 
ering to ensure that they cover concepts 
involving rational numbers in depth. The 
focus on rational numbers should include 
understanding the meaning of fractions, 
decimals, ratios, and percents, using visual 
representations (including placing fractions 
and decimals on number lines,^® see recom- 
mendation 5), and solving problems with 
fractions, decimals, ratios, and percents. 

In the view of the panel, students in 
grades 4 through 8 will also require ad- 
ditional work to build fluent retrieval of 
basic arithmetic facts (see recommenda- 
tion 6), and some will require additional 
work involving basic whole number top- 
ics, especially for students in tier 3. In the 
opinion of the panel, accurate and fluent 



48. When using number lines to teach rational 
numbers for students who have difficulties, it is 
important to emphasize that the focus is on the 
length of the segments between the whole num- 
ber marks (rather than counting the marks). 
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arithmetic with whole numbers is neces- 
sary before understanding fractions. The 
panel acknowledges that there will be 
periods when both whole numbers and 
rational numbers should be addressed in 
interventions. In these cases, the balance 
of concepts should be determined by the 
student’s need for support. 



3. Districts should appoint committees, in- 
cluding experts in mathematics instruction 
and mathematicians with knowledge of el- 
ementary and middle school mathematics 
curriculum, to ensure that specific criteria 
(described below) are covered in depth in 
the curricula they adopt. 

In the panel’s view, intervention materials 
should be reviewed by individuals with 
knowledge of mathematics instruction and 
by mathematicians knowledgeable in el- 
ementary and middle school mathematics. 
They can often be experts within the district, 
such as mathematics coaches, mathematics 
teachers, or department heads. Some dis- 
tricts may also be able to draw on the exper- 
tise of local university mathematicians. 

Reviewers should assess how well interven- 
tion materials meet four criteria. First, the 
materials integrate computation with solv- 
ing problems and pictorial representations 
rather than teaching computation apart 
from problem-solving. Second, the mate- 
rials stress the reasoning underlying cal- 
culation methods and focus student atten- 
tion on making sense of the mathematics. 
Third, the materials ensure that students 
build algorithmic proficiency. Fourth, the 
materials include frequent review for both 
consolidating and understanding the links 
of the mathematical principles. Also in the 



panel’s view, the intervention program 
should include an assessment to assist in 
placing students appropriately in the in- 
tervention curriculum. 

Potential roadblocks and solutions 

Roadblock 2.1. Some interventionists 
may worry if the intervention program 
is not aligned with the core classroom 
instruction. 

Suggested Approach. The panel believes 
that alignment with the core curriculum is 
not as critical as ensuring that instruction 
builds students’ foundational proficien- 
cies. Tier 2 and tier 3 instruction focuses 
on foundational and often prerequisite 
skills that are determined by the students’ 
rate of progress. So, in the opinion of the 
panel, acquiring these skills will be neces- 
sary for future achievement. Additionally, 
because tier 2 and tier 3 are supplemental, 
students will still be receiving core class- 
room instruction aligned to a school or 
district curriculum (tier 1). 

Roadblock 2.2. Intervention materials 
may cover topics that are not essential to 
building basic competencies, such as data 
analysis, measurement, and time. 

Suggested Approach. In the panel’s opin- 
ion, it is not necessary to cover every topic 
in the intervention materials. Students will 
gain exposure to many supplemental top- 
ics (such as data analysis, measurement, 
and time) in general classroom instruc- 
tion (tier 1). Depending on the student’s 
age and proficiency, it is most important 
to focus on whole and rational numbers in 
the interventions. 
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Recommendation 3. 
Instruction during the 
intervention should be 
explicit and systematic. 
This includes providing 
models of proficient 
problem solving, 
verbalization of 
thought processes, 
guided practice, 
corrective feedback, 
and frequent 
cumulative review. 

The National Mathematics Advisory 
Panel defines explicit instruction as 
follows (2008, p. 23): 

• “Teachers provide clear models for 
solving a problem type using an 
array of examples.” 

• “Students receive extensive practice 
in use of newly learned strategies 
and skills.” 

• “Students are provided with 
opportunities to think aloud (i.e., 
talk through the decisions they 
make and the steps they take).” 

• “Students are provided with 
extensive feedback.” 

The NMAP notes that this does not mean 
that all mathematics instruction should 
be explicit. But it does recommend that 
struggling students receive some explicit 
instruction regularly and that some 
of the explicit instruction ensure that 
students possess the foundational skills 
and conceptual knowledge necessary 
for understanding their grade-level 



mathematics."^^ Our panel supports 
this recommendation and believes 
that districts and schools should select 
materials for interventions that reflect 
this orientation. In addition, professional 
development for interventionists should 
contain guidance on these components 
of explicit instruction. 

Level of evidence: Strong 

Our panel judged the level of evidence 
supporting this recommendation to be 
strong. This recommendation is based on 
six randomized controlled trials that met 
WWC standards or met standards with 
reservations and that examined the ef- 
fectiveness of explicit and systematic in- 
struction in mathematics interventions.^® 
These studies have shown that explicit and 
systematic instruction can significantly 
improve proficiency in word problem solv- 
ing^i and operations'^ across grade levels 
and diverse student populations. 

Brief summary of evidence to support 
the recommendation 

The results of six randomized controlled 
trials of mathematics interventions show 
extensive support for various combina- 
tions of the following components of ex- 
plicit and systematic instruction: teacher 
demonstration, student verbalization,^^ 



49. National Mathematics Advisory Panel 
(2008). 

50. Darch, Carnine, and Gersten (1984); Fuchs 
et al. (2003a); Jitendra et al. (1998); Schunk and 
Cox (1986); Tournaki (2003); Wilson and Sindelar 
(1991). 

51. Darch, Carnine, and Gersten (1984); Jitendra 
et al. (1998); Fuchs et al. (2003a); Wilson and Sin- 
delar (1991). 

52. Schunk and Cox (1986); Tournaki (2003). 

53. Darch, Carnine, and Gersten (1984); Jitendra 
et al. (1998); Fuchs et al. (2003a); Schunk and 
Cox (1986); Tournaki (2003); Wilson and Sindelar 
(1991). 

54. Jitendra et al. (1998); Fuchs et al. (2003a); 
Schunk and Cox (1986); Tournaki (2003). 
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guided practice, and corrective feed- 
back.^® All six studies examined interven- 
tions that included teacher demonstra- 
tions early in the lessons. For example, 
three studies included instruction that 
began with the teacher verbalizing aloud 
the steps to solve sample mathematics 
problems.^® The effects of this component 
of explicit instruction cannot be evaluated 
from these studies because the demonstra- 
tion procedure was used in instruction for 
students in both treatment and compari- 
son groups. 

Scaffolded practice, a transfer of control 
of problem solving from the teacher to the 
student, was a component in four of the six 
studies.®® Although it is not possible to parse 
the effects of scaffolded instruction from the 
other components of instruction, the inter- 
vention groups in each study demonstrated 
significant positive gains on word problem 
proficiencies or accuracy measures. 

Three of the six studies included opportu- 
nities for students to verbalize the steps 
to solve a problem.®® Again, although ef- 
fects of the interventions were statistically 
significant and positive on measures of 
word problems, operations, or accuracy, 
the effects cannot be attributed to a sin- 
gle component of these multicomponent 
interventions. 



55. Darch, Carnine, and Gersten (1984); Jiten- 
dra et al. (1998); Fuchs et al. (2003a); Tournaki 
(2003). 

56. Darch, Carnine, and Gersten (1984); Jitendra 
et al. (1998); Schunk and Cox (1986); Tournaki 
(2003). 

57. Darch, Carnine, and Gersten (1984); Fuchs 
et al. (2003a); Jitendra et al. (1998); Schunk and 
Cox (1986); Tournaki (2003); Wilson and Sindelar 
(1991). 

58. Schunk and Cox (1986); Jitendra et al. (1998); 
Darch, Carnine, and Gersten (1984). 

59. Darch, Carnine, and Gersten (1984); Fuchs 
et al. (2003a); Jitendra et al. (1998); Tournaki 
(2003). 

60. Schunk and Cox (1986); Jitendra et al. (1998); 
Tournaki (2003). 



Similarly, four of the six studies included 
immediate corrective feedback,®^ and the 
effects of these interventions were posi- 
tive and significant on word problems and 
measures of operations skills, but the ef- 
fects of the corrective feedback compo- 
nent cannot be isolated from the effects of 
other components in three cases. ®^ 

With only one study in the pool of six in- 
cluding cumulative review as part of the 
intervention,®® the support for this compo- 
nent of explicit instruction is not as strong 
as it is for the other components. But this 
study did have statistically significant pos- 
itive effects in favor of the instructional 
group that received explicit instruction 
in strategies for solving word problems, 
including cumulative review. 

How to carry out this 
recommendation 



1. Ensure that instructional materials are 
systematic and explicit. In particular, they 
should include numerous clear models of 
easy and difficult problems, with accompa- 
nying teacher think-alouds. 

To be considered systematic, mathematics 
instruction should gradually build profi- 
ciency by introducing concepts in a logical 
order and by providing students with nu- 
merous applications of each concept. For 
example, a systematic curriculum builds 
student understanding of place value in 
an array of contexts before teaching pro- 
cedures for adding and subtracting two- 
digit numbers with regrouping. 

Explicit instruction typically begins with 
a clear unambiguous exposition of con- 
cepts and step-by-step models of how 



61. Darch, Carnine, and Gersten (1984); Jiten- 
dra et al. (1998); Tournaki (2003); Schunk and 
Cox (1986). 

62. Darch, Carnine, and Gersten (1984); Jitendra 
et al. (1998); Tournaki (2003). 

63. Fuchs et al. (2003a). 
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to perform operations and reasons for 
the procedures.®'^ Interventionists should 
think aloud (make their thinking pro- 
cesses public) as they model each step of 
the process.®^’®® They should not only tell 
students about the steps and procedures 
they are performing, but also allude to the 
reasoning behind them (link to the under- 
lying mathematics). 

The panel suggests that districts select 
instructional materials that provide inter- 
ventionists with sample think-alouds or 
possible scenarios for explaining concepts 
and working through operations. A crite- 
rion for selecting intervention curricula 
materials should be whether or not they 
provide materials that help intervention- 
ists model or think through difficult and 
easy examples. 

In the panel’s view, a major flaw in many 
instructional materials is that teachers are 
asked to provide only one or two models 
of how to approach a problem and that 
most of these models are for easy-to-solve 
problems. Ideally, the materials will also 
assist teachers in explaining the reason- 
ing behind the procedures and problem- 
solving methods. 



2. Provide students with opportunities to 
solve problems in a group and communicate 
problem-solving strategies. 

For students to become proficient in per- 
forming mathematical processes, explicit 
instruction should include scaffolded prac- 
tice, where the teacher plays an active 
role and gradually transfers the work to 



64. For example, Jitendra et al. (1998); Darch, 
Carnine, and Gersten (1984); Woodward (2006). 

65. See an example in the summary of Tournaki 
(2003) in appendix D. 

66. Darch, Carnine, and Gersten (1984); Jiten- 
dra et al. (1998); Fuchs et al. (2003a); Schunk 
and Cox (1986); Tournaki (2003); Wilson and Sin- 
delar (1991). 



the students.®^ This phase of explicit in- 
struction begins with the teacher and the 
students solving problems together. As 
this phase of instruction continues, stu- 
dents should gradually complete more 
steps of the problem with decreasing guid- 
ance from the teacher. Students should 
proceed to independent practice when 
they can solve the problem with little or 
no support from the teacher. 

During guided practice, the teacher should 
ask students to communicate the strate- 
gies they are using to complete each step 
of the process and provide reasons for 
their decisions.®® In addition, the panel 
recommends that teachers ask students to 
explain their solutions.®® Note that not only 
interventionists — but fellow students — can 
and should communicate how they think 
through solving problems to the inter- 
ventionist and the rest of the group. This 
can facilitate the development of a shared 
language for talking about mathematical 
problem solving.^® 

Teachers should give specific feedback 
that clarifies what students did correctly 
and what they need to improve. They 
should provide opportunities for students 
to correct their errors. For example, if a 
student has difficulty solving a word prob- 
lem or solving an equation, the teacher 
should ask simple questions that guide the 
student to solving the problem correctly. 
Corrective feedback can also include re- 
teaching or clarifying instructions when 
students are not able to respond to ques- 
tions or their responses are incorrect. 



67. Tournaki (2003); Jitendra et al. (1998); Darch, 
Carnine, and Gersten (1984). 

68. For example, Schunk and Cox (1986). 

69. Schunk and Cox (1986); Tournaki (2003). 

70. For example, Jitendra et al. (1998); Darch, 
Carnine, and Gersten (1984). 

71. Tournaki (2003); Jitendra et al. (1998); Darch, 
Carnine, and Gersten (1984). 
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3. Ensure that instructional materials include 
cumulative review in each session. 

Cumulative reviews provide students with 
an opportunity to practice topics previ- 
ously covered in depth. For example, when 
students are working with fractions, a 
cumulative review activity could provide 
them with an opportunity to solve some 
problems involving multiplication and di- 
vision of whole numbers. In the panel’s 
opinion, this review can ensure that the 
knowledge is maintained over time and 
helps students see connections between 
various mathematical ideas. 

Potential roadblocks and solutions 

Roadblock 3.1. Interventionists maybe un- 
familiar with how to implement an interven- 
tion that uses explicit instruction, and some 
may underestimate the amount of practice 
necessary for students in tiers 2 and 3 to 
master the material being taught. 

Suggested Approach. Districts and 
schools should set up professional devel- 
opment sessions for interventionists to 
observe and discuss sample lessons. The 
panel believes that it is important for pro- 
fessional development participants to ob- 
serve the intervention first hand. Watching 
a DVD or video of the intervention being 
used with students can give the partici- 
pants a model of how the program should 
be implemented. 

Interventionists should also have hands- 
on experience, teaching the lessons to 
each other and practicing with students. 
Role-playing can give interventionists 
practice with modeling and think-alouds, 
since it is important for them to stop and 
reflect before formulating an explanation 
for their thinking processes. The train- 
ers can observe these activities, provide 
feedback on what participants did well, 
and offer explicit suggestions for improv- 
ing instruction. 



As a part of professional development, be 
sure to convey the benefits that extended 
practice (not only worksheets) and cumu- 
lative review can have for student per- 
formance. If professional development 
is not an option, teachers can also work 
with mathematics coaches to learn how to 
implement the intervention. 

Roadblock 3.2. Interventionists may not 
be expert with the underlying mathemat- 
ics content. 

Suggested Approach. For intervention- 
ists to explain a mathematical process ac- 
curately and develop a logical think-aloud, 
it is important for them to understand the 
underlying mathematics concept and the 
mathematical reasoning for the process. 
Professional development should provide 
participants with in-depth knowledge of 
the mathematics content in the interven- 
tion, including the mathematical reason- 
ing underlying procedures, formulas, and 
problem-solving methods. The panel be- 
lieves that when interventionists convey 
their knowledge of the content, student 
understanding will increase, misconcep- 
tions will decrease, and the chances that 
students solve problems by rote memory 
will be reduced. 

Roadblock 3.3. The intervention materials 
may not incorporate enough models, think- 
alouds, practice, and cumulative review. 

Suggested Approach. Intervention pro- 
grams might not incorporate enough mod- 
els, think-alouds, practice, or cumulative 
review to improve students’ mathematics 
performance.^^ 

Consider using a mathematics coach or 
specialist to develop a template listing 
the essential parts of an effective lesson. 



72. National Mathematics Advisory Panel (2008); 
Wu (2005) http://math.berkeley.edu/~wu/ 
Northridge2004a2.pdf. 

73. Jitendra et al. (1996); Carnine et al. (1997). 
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including the number of models, accom- 
panying think-alouds, and practice and 
cumulative review items students need to 
understand, learn, and master the content. 



A team of teachers, guided by the math- 
ematics coach/specialist, can determine 
the components that should be added to 
the program. 
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Recommendation 4. 
Interventions should 
include instruction 
on solving word 
problems that is 
based on common 
underlying structures. 

Students who have difficulties in 
mathematics typically experience severe 
difficulties in solving word problems 
related to the mathematics concepts and 
operations they are learning/"^ This is a 
major impediment for future success in 
any math-related discipline/^ 

Based on the importance of building 
proficiency and the convergent findings 
from a body of high-quality research, 
the panel recommends that interventions 
include systematic explicit instruction 
on solving word problems, using 
the problems’ underlying structure. 
Simple word problems give meaning 
to mathematical operations such as 
subtraction or multiplication. When 
students are taught the underlying 
structure of a word problem, they not 
only have greater success in problem 
solving but can also gain insight into 
the deeper mathematical ideas in word 
problems.^® The panel also recommends 
systematic instruction on the structural 
connections between known, familiar 
word problems and unfamiliar, new 
problems. By making explicit the 
underlying structural connections 
between familiar and unfamiliar problems, 
students will know when to apply the 
solution methods they have learned. 



74. Geary (2003); Hanich et al. (2001). 

75. National Mathematics Advisory Panel (2008); 
McCloskey (2007). 

76. Peterson, Fennema, and Carpenter (1989). 

77. Fuchs, Fuchs, Finelli et al. (2004). 



Level of evidence: Strong 

The panel judged the level of evidence 
supporting this recommendation to be 
strong. This recommendation is based 
on nine randomized controlled trials that 
met WWC standards or met standards 
with reservations and that examined the 
effectiveness of word problem-solving 
strategies.^® Interventions that teach stu- 
dents the structure of problem types^^ — 
and how to discriminate superficial from 
substantive information to know when 
to apply the solution methods they have 
learned®° — positively and marginally or 
significantly affect proficiency in solving 
word problems. 

Brief summary of evidence to 
support the recommendation 

Research demonstrates that instruction on 
solving word problems based on under- 
lying problem structure leads to statisti- 
cally significant positive effects on mea- 
sures of word problem solving. Three 
randomized controlled trials isolated this 
practice. In these studies, intervention- 
ists taught students to identify problems 
of a given type by focusing on the prob- 
lem structure and then to design and 
execute appropriate solution strategies 
for each problem. These techniques typi- 
cally led to significant and positive effects 
on word-problem outcomes for students 



78. Jitendra et al. (1998); Xin, Jitendra, and Deat- 
line-Buchman (2005); Darch, Gamine, and Gersten 
(1984); Fuchs et al. (2003a); Fuchs et al. (2003b); 
Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, 
Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. 
(2008); Fuchs, Seethaler et al. (2008). 

79. Jitendra et al. (1998); Xin, jitendra, and Deat- 
line-Buchman (2005); Darch, Gamine, and Ger- 
sten (1984). 

80. Fuchs et al. (2003a); Fuchs et al. (2003b); 
Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, 
Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. 
(2008); Fuchs, Seethaler et al. (2008). 

81. Jitendra et al. (1998); Xin, Jitendra, and Deat- 
line-Buchman (2005); Darch, Gamine, and Ger- 
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experiencing difficulties in mathematics How to carry out this 
across grade levels. recommendation 



Six other randomized controlled trials 
took the instructional intervention on 
problem structure a step further. They 
demonstrated that teaching students to 
distinguish superficial from substantive 
information in problems also leads to 
marginally or statistically significant posi- 
tive effects on measures of word problem 
solving.®^ After students were explicitly 
taught the pertinent structural features 
and problem-solution methods for differ- 
ent problem types, they were taught su- 
perficial problem features that can change 
a problem without altering its underlying 
structure. They were taught to distinguish 
substantive information from superficial 
information in order to solve problems 
that appear new but really fit into one of 
the categories of problems they already 
know how to solve. They were also taught 
that the same underlying problem struc- 
tures can be applied to problems that 
are presented in graphic form (for exam- 
ple, with tables or maps). These are pre- 
cisely the issues that often confuse and 
derail students with difficulties in math- 
ematics. These six studies consistently 
demonstrated marginally or statistically 
significant positive effects on an array 
of word problem-solving proficiencies 
for students experiencing difficulties in 
mathematics.®^ 



82. Jitendra et al. (1998); Xin, Jitendra, and Deat- 
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(2008); Fuchs, Seethaler et al. (2008). 
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1. Teach students about the structure of 
various problem types, how to categorize 
problems based on structure, and how to 
determine appropriate solutions for each 
problem type. 



Students should be explicitly taught about 
the salient underlying structural features 
of each problem type.®® Problem types are 
groups of problems with similar math- 
ematical structures. For example, change 
problems describe situations in which a 
quantity (such as children or pencils) is 
either increased or decreased (example 1). 
Change problems always include a time 
element. For these problems, students 
determine whether to add or subtract by 
determining whether the change in the 
quantity is more or less. 



Example 1. Change problems 



The two problems here are addition and 
subtraction problems that students may 
be tempted to solve using an incorrect op- 
eration. In each case, students can draw a 
simple diagram like the one shown below, 
record the known quantities (two of three 
of A, B, and C) and then use the diagram to 
decide whether addition or subtraction is 
the correct operation to use to determine 
the unknown quantity. 




C 

Problem 1. Brad has a bottlecap collection. 
After Madhavi gave Brad 28 more bottle- 
caps, Brad had 111 bottlecaps. How many 
bottlecaps did Brad have before Madhavi 
gave him more? 

Problem 2. Brad has a bottlecap collection. 
After Brad gave 28 of his bottlecaps to Mad- 
havi, he had 83 bottlecaps left. How many 
bottlecaps did Brad have before he gave 
Madhavi some? 



85. Xin, Jitendra, and Deatline-Buchman (2005). 
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In contrast, compare problems have no 
time element (example 2). They focus on 
comparisons between two different types 
of items in two different sets (pears and 
apples, boys and girls, hot and cold items). 
Students add or subtract by determin- 
ing whether they need to calculate the 
unknown difference (subtract), unknown 
compared amount (add), or unknown ref- 
erent amount (subtract). 

Example 2. Compare problems 



There are 21 hamsters and 32 kittens at 
the pet store. How many more kittens are 
at the pet store than hamsters? 

32 

21 ; ? ; 



Although these problem types seem simple 
and intuitive to adults and mathematically 
precocious students, they are not neces- 
sarily obvious for students requiring math- 
ematics interventions. To build understand- 
ing of each problem type, we recommend 
initially teaching solution rules (or guiding 
questions that lead to a solution equation) 
for each problem type through fully and 
partially worked examples, followed by 
student practice in pairs. 

Visual representations such as those in ex- 
ample 2 can be effective for teaching stu- 
dents how to categorize problems based 
on their structure and determine a solu- 
tion method appropriate for the underlying 
structure (see recommendation 5 for more 
information on visual representations).®^ 
Teachers can present stories with unknown 
information and work with students in 
using diagrams to identify the problem 
type and transform the information in the 
diagram into a mathematics equation to 
solve for the unknown quantity. 



86. Fuchs, Fuchs, Finelli et al. (2004). 
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2. Teach students to recognize the common 
underlying structure between familiar and 
unfamiliar problems and to transfer known 
solution methods from familiar to unfamil- 
iar problems. 

A known familiar problem often appears 
as a new and unfamiliar problem to a stu- 
dent because of such superficial changes 
as format changes (whether it is written in 
traditional paragraph form or as an adver- 
tisement for a brochure), key vocabulary 
changes (half, one-half, /4), or the inclusion 
of irrelevant information (additional story 
elements such as the number of buttons 
on a child’s shirt or the size of a storage 
container for a compare problem).®® These 
superficial changes are irrelevant to un- 
derstanding the mathematical demands 
of a problem. But while focusing on these 
irrelevant superficial changes, students 
can find it difficult to discern the critical 
common underlying structure between the 
new and the old problems and to apply the 
solution that is part of their repertoire to 
the new unfamiliar problem. 

To facilitate the transfer of the known so- 
lution from the familiar to the unfamiliar 
problem, students should first be shown 
explicitly that not all pieces of information 
in the problem are relevant to discerning 
the underlying problem structure.®® Teach- 
ers should explain these irrelevant superfi- 
cial features explicitly and systematically, 
as described in recommendation 3.®° This 
instruction may be facilitated by the use 
of a poster displayed in the classroom that 
lists the ways familiar problems can be- 
come unfamiliar because of new wording or 
situations (such as information displayed in 
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RECOMMENDATION 4. INTERVENTIONS SHOULD INCLUDE INSTRUCTION ON SOLVING WORD PROBLEMS 



chart versus paragraph form) or the ways 
relevant to problem type. The students 
must also be provided with opportunities 
to explain why a piece of information is 
relevant or irrelevant.^^ 

We suggest that students practice sets of 
problems with varied superficial features 
and cover stories. Students who know how 
to recognize and solve a “change” problem 
type with whole numbers should know 
that they can apply the same strategy 
to a structurally similar word problem 
that looks different because of changes in 
wording and the presence of additional 
story elements (example 3).^^ 



Potential roadblocks and solutions 

Roadblock 4.1. In the opinion of the panel, 
the curricular material may not classify 
problems into problem types. 

Suggested Approach. The interventionist 
may need the help of a mathematics coach, 
a mathematics specialist, or a district or 
state curriculum guide in determining 
the problem types and an instructional 
sequence for teaching them to students. 
The key issue is that students are taught 
to understand a set of problem structures 
related to the mathematics they are learn- 
ing in their intervention. 



Example 3. Solving different 
problems with the same strategy 



■ Mike wants to buy 1 pencil for each 
of his friends. Each packet of pencils 
contains 12 pencils. How many pack- 
ets does Mike have to buy to give 1 
pencil to each of his 13 friends? 

■ Mike wants to buy 1 pencil for each of 
his friends. Sally wants to buy 1 0 pen- 
cils. Each box of pencils contains 12 
pencils. How many boxes does Mike 
have to buy to give 1 pencil to each 
of his 13 friends? 
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Roadblock 4.2. As problems get complex, 
so will the problem types and the task of 
discriminating among them. 

Suggested Approach. As problems get 
more intricate (such as multistep problems), 
it becomes more difficult for students to 
determine the problem type, a critical step 
that leads to solving the problem correctly. 
It is important to explicitly and systemati- 
cally teach students how to differentiate 
one problem type from another. 

Interventionists will need high-quality 
professional development to ensure that 
they convey the information clearly and 
accurately. The professional development 
program should include opportunities for 
participants to determine problem types, 
justify their responses, and practice ex- 
plaining and modeling problem types to 
peers and children. Trainers should pro- 
vide constructive feedback during the 
practice sessions by telling participants 
both what they did well and what aspects 
of their instruction need improvement. 
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Recommendation 5. 
Intervention materials 
should include 
opportunities for 
students to work with 
visual representations 
of mathematical ideas 
and interventionists 
should be proficient 
in the use of visual 
representations of 
mathematical ideas. 

A major problem for students who 
struggle with mathematics is weak 
understanding of the relationships 
between the abstract symbols 
of mathematics and the various 
visual representations.^^ Student 
understanding of these relationships 
can be strengthened through the 
use of visual representations of 
mathematical concepts such as solving 
equations, fraction equivalence, and 
the commutative property of addition 
and multiplication (see the glossary). 
Such representations may include 
number lines, graphs, simple drawings 
of concrete objects such as blocks or 
cups, or simplified drawings such as 
ovals to represent birds. 

In the view of the panel, the ability 
to express mathematical ideas using 
visual representations and to convert 
visual representations into symbols is 
critical for success in mathematics. A 
major goal of interventions should be 
to systematically teach students how 
to develop visual representations and 
how to transition these representations 
to standard symbolic representations 
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used in problem solving. Occasional 
and unsystematic exposure (the norm 
in many classrooms) is insufficient and 
does not facilitate understanding of 
the relationship between the abstract 
symbols of mathematics and various 
visual representations. 

Level of evidence: Moderate 

The panel judged the level of evidence sup- 
porting this recommendation to be mod- 
erate. This recommendation is based on 
13 randomized controlled trials that met 
WWC standards or met standards with 
reservations. These studies provide sup- 
port for the systematic use of visual rep- 
resentations or manipulatives to improve 
achievement in general mathematics,^^ 
prealgebra concepts,^® word problems,®^ 
and operations.®® But these representations 
were part of a complex multicomponent 
intervention in each of the studies. So, it is 
difficult to judge the impact of the repre- 
sentation component alone, and the panel 
believes that a moderate designation is ap- 
propriate for the level of evidence for this 
recommendation. 

Brief summary of evidence to 
support the recommendation 

Research shows that the systematic use of 
visual representations and manipulatives 
may lead to statistically significant or sub- 
stantively important positive gains in math 
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RECOMMENDATION 5. INTERVENTION MATERIALS SHOULD INCLUDE OPPORTUNITIES EOR THE STUDENT 



achievement. Four studies used visual 
representations to help pave the way for 
students to understand the abstract version 
of the representation. For example, one 
of the studies taught students to use visual 
representations such as number lines to 
understand mathematics facts. The four 
studies demonstrated gains in mathematics 
facts and operations^°^ and word problem 
proficiencies, and may provide evidence 
that using visual representations in inter- 
ventions is an effective technique. 

Three of the studies used manipulatives 
in the early stages of instruction to rein- 
force understanding of basic concepts and 
operations. One used concrete models 
such as groups of boxes to teach rules for 
multiplication problems. The three stud- 
ies largely showed significant and positive 
effects and provide evidence that using 
manipulatives may be helpful in the initial 
stages of an intervention to improve profi- 
ciency in word problem solving. 

In six of the studies, both concrete and vi- 
sual representations were used, and over- 
all these studies show that using some 
combination of manipulatives and visual 
representations may promote mathemati- 
cal understanding. In two of the six, 
instruction did not include fading of the 
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manipulatives and visual representations 
to promote understanding of math at a 
more abstract level. One of these inter- 
ventions positively affected general math 
achievement, but the other had no effect 
on outcome measures tested. In the other 
four studies, manipulatives and visual rep- 
resentations were presented to the students 
sequentially to promote understanding 
at a more abstract level. One interven- 
tion that used this method for teaching 
fractions did not show much promise, 
but the other three did result in positive 
gains. One of them taught 1st graders 
basic math concepts and operations, and 
the other two taught prealgebra concepts 
to low-achieving students. 

How to carry out this 
recommendation 



1. Use visual representations such as number 
lines, arrays, and strip diagrams. 

In the panel’s view, visual representations 
such as number lines, number paths, strip 
diagrams, drawings, and other forms of pic- 
torial representations help scaffold learn- 
ing and pave the way for understanding the 
abstract version of the representation. We 
recommend that interventionists use such 
abstract visual representations extensively 
and consistently. We also recommend that 
interventionists explicitly link visual rep- 
resentations with the standard symbolic 
representations used in mathematics. 
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RECOMMENDATION 5. INTERVENTION MATERIALS SHOULD INCLUDE OPPORTUNITIES FOR THE STUDENT 



In early grades, number lines, number 
paths, and other pictorial representations 
are often used to teach students founda- 
tional concepts and procedural operations 
of addition and subtraction. Although 
number lines or number paths may not be 
a suitable initial representation in some 
situations (as when working with multipli- 
cation and division), they can help concep- 
tually and procedurally with other types of 
problems. Conceptually, number lines and 
number paths show magnitude and allow 
for explicit instruction on magnitude com- 
parisons. Procedurally, they help teach 
principles of addition and subtraction op- 
erations such as “counting down,” “count- 
ing up,” and “counting down from.” 

The figure in example 4 shows how a num- 
ber line may be used to assist with counting 
strategies. The top arrows show how a child 

learns to count on. He adds 2 + 5 = . To 

start, he places his finger on 2. Then, he 
jumps five times to the right and lands on 
7. The arrows under the number line show 
how a child subtracts using a counting 

down strategy. For 10 - 3 = , she starts 

with her finger on the 10. Then, she jumps 
three times to the left on the number line, 
where she finishes on 7. 

The goal of using a number line should 
be for students to create a mental num- 
ber line and establish rules for movement 
along the line according to the more or less 
marking arrows placed along the line. Such 
rules and procedures should be directly 
tied to the explicit instruction that guided 
the students through the use of the visual 
representation.^^® 

Pictorial representations of objects such 
as birds and cups are also often used to 
teach basic addition and subtraction, and 
simple drawings can help students under- 
stand place value and multidigit addition 
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and subtraction. Example 5 (p. 34) shows 
how a student can draw a picture to solve 
a multidigit addition problem. In the fig- 
ure, circles represent one unit and lines 
represent units of 10. 

In upper grades, diagrams and pictorial 
representations used to teach fractions 
also help students make sense of the basic 
structure underlying word problems. Strip 
diagrams (also called model diagrams and 
bar diagrams) are one type of diagram that 
can be used. Strip diagrams are drawings of 
narrow rectangles that show relationships 
among quantities. Students can use strip 
diagrams to help them reason about and 
solve a wide variety of word problems about 
related quantities. In example 6 (p. 34), the 
full rectangle (consisting of all three equal 
parts joined together) represents Shauntay’s 
money before she bought the book. Since 
she spent % of her money on the book, two 
of the three equal parts represent the $26 
she spent on the book. Students can then 
reason that if two parts stand for $26, then 
each part stands for $13, so three parts 
stand for $39. So, Shauntay had $39 before 
she bought the book. 



2. If visuals are not sufficient for developing 
accurate abstract thought and answers, use 
concrete manipulatives first. Although this 
can also be done with students in upper el- 
ementary and middle school grades, use of 
manipulatives with older students should be 
expeditious because the goal is to move to- 
ward understanding of— and facility with — 
visual representations, and finally, to the 
abstract. 

Manipulatives are usually used in lower 
grades in the initial stages of learning as 
teachers introduce basic concepts with 
whole numbers. This exposure to concrete 
objects is often fleeting and transitory. The 
use of manipulatives in upper elementary 
school grades is virtually nonexistent."’ 
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RECOMMENDATION 5. INTERVENTION MATERIALS SHOULD INCLUDE OPPORTUNITIES EOR THE STUDENT 



The panel suggests that the interventionist 
use concrete objects in two ways. 

First, in lower elementary grades, use 
concrete objects more extensively in the 
initial stages of learning to reinforce the 
understanding of basic concepts and 
operations."* 

Concrete models are routinely used to 
teach basic foundational concepts such 
as place value. They are also useful in 
teaching other aspects of mathematics 
such as multiplication facts. When a mul- 
tiplication fact is memorized by question 
and answer alone, a student may believe 
that numbers are to be memorized rather 
than understood. For example, 4x6 equals 
24. When shown using manipulatives (as 
in example 7, p. 35), 4x6 means 4 groups 
of 6, which total as 24 objects. 
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Second, in the upper grades, use concrete 
objects when visual representations do 
not seem sufficient in helping students 
understand mathematics at the more ab- 
stract level. 

Use manipulatives expeditiously, and focus 
on fading them away systematically to 
reach the abstract level. In other words, 
explicitly teach students the concepts and 
operations when students are at the con- 
crete level and consistently repeat the in- 
structional procedures at the visual and 
abstract levels. Using consistent language 
across representational systems (manip- 
ulatives, visual representations, and ab- 
stract symbols) has been an important 
component in several research studies. 
Example 8 (p. 35) shows a set of matched 
concrete, visual, and abstract representa- 
tions of a concept involving solving single- 
variable equations. 
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Example 4. Representation of the counting on strategy using 
a number line 




23456789 10 
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Example 5. Using visual representations for multidigit addition 



A group often can be 
drawn with a long line 
to indicate that ten ones 
are joined to form one ten: 




Simple drawings 
help make sense 
of two-digit addition 
with regrouping: 




Example 6. Strip diagrams can help students make sense of fractions 



Shauntay spent % 
of the money she had 
on a book that cost $26. 
How much money did 
Shauntay have before 
she bought the book? 



Shauntay’s money at first 




$26 book 



2 parts 
1 part 

3 parts 



► $26 

► $26 - 2 = $13 

► 3 X $13 = $39 



Shauntay’s had $39 
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Example 7. Manipulatives can help students understand that four multiplied 
by six means four groups of six, which means 24 total objects 




Example 8. A set of matched concrete, visual, and abstract representations to teach 
solving single-variable equations 



3 + X=7 



Solving the Equation with 
Concrete Manipulatives 
(Cups and Sticks) 



Solving the Equation 
with Visual 
Representations 
of Cups and Sticks 



Solving the Equation 
with Abstract Symbols 






X 



4 



Concrete Steps 

A. 3 sticks plus one group of X equals 7 sticks 

B. Subtract 3 sticks from each side of the equation 

C. The equation now reads as one group of X equals 4 sticks 

D. Divide each side of the equation by one group 

E. One group of X is equal to four sticks (i.e., 1 X/group = 4 sticks/group; 1 X = 4 sticks) 
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RECOMMENDATION 5. INTERVENTION MATERIALS SHOULD INCLUDE OPPORTUNITIES FOR THE STUDENT 



Potential roadblocks and solutions 

Roadblock 5.1. In the opinion of the 
panel, many intervention materials pro- 
vide very few examples of the use of visual 
representations. 

Suggested Approach. Because many cur- 
ricular materials do not include sufficient 
examples of visual representations, the 
interventionist may need the help of the 
mathematics coach or other teachers in 
developing the visuals. District staff can 
also arrange for the development of these 
materials for use throughout the district. 

Roadblock 5.2. Some teachers or interven- 
tionists believe that instruction in concrete 
manipulatives requires too much time. 

Suggested Approach. Expeditious use of 
manipulatives cannot be overemphasized. 
Since tiered interventions often rely on 
foundational concepts and procedures, 
the use of instruction at the concrete level 
allows for reinforcing and making explicit 
the foundational concepts and operations. 
Note that overemphasis on manipulatives 
can be counterproductive, because stu- 
dents manipulating only concrete objects 
may not be learning to do math at an ab- 
stract level. The interventionist should 
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use manipulatives in the initial stages stra- 
tegically and then scaffold instruction to 
the abstract level. So, although it takes time 
to use manipulatives, this is not a major 
concern since concrete instruction will 
happen only rarely and expeditiously. 

Roadblock 5.3. Some interventionists 
may not fully understand the mathemati- 
cal ideas that underlie some of the repre- 
sentations. This is likely to be particularly 
true for topics involving negative numbers, 
proportional reasoning, and interpretations 
of fractions. 

Suggested Approach. If interventionists 
do not fully understand the mathematical 
ideas behind the material, they are un- 
likely to be able to teach it to struggling 
students. It is perfectly reasonable for 
districts to work with a local university 
faculty member, high school mathemat- 
ics instructor, or mathematics special- 
ist to provide relevant mathematics in- 
struction to interventionists so that they 
feel comfortable with the concepts. This 
can be coupled with professional devel- 
opment that addresses ways to explain 
these concepts in terms their students will 
understand. 
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Recommendation 6. 
Interventions at all 
grade levels should 
devote about 10 minutes 
in each session to 
building fluent retrieval 
of basic arithmetic facts. 

Quick retrieval of basic arithmetic 
facts is critical for success in 
mathematics.'^"^ Yet research has found 
that many students with difficulties 
in mathematics are not fluent in 
such facts.'^^ Weak ability to retrieve 
arithmetic facts is likely to impede 
understanding of concepts students 
encounter with rational numbers 
since teachers and texts often assume 
automatic retrieval of facts such as 

3x9 = and 11-7 = as they 

explain concepts such as equivalence 
and the commutative property.'^^ For 
that reason, we recommend that about 
10 minutes be devoted to building this 
proficiency during each intervention 
session. Acknowledging that time may 
be short, we recommend a minimum 
of 5 minutes a session. 

Level of evidence: Moderate 

The panel judged the level of evidence 
supporting this recommendation to be 
moderate. This recommendation is based 
on seven randomized controlled trials that 
met WWC standards or met standards with 
reservations and that included fact fluency 
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instruction in the intervention.'^^ These 
studies reveal a series of small but positive 
effects on measures of fact fluency'^® and 
procedural knowledge for diverse student 
populations in the elementary grades. 

In some cases, fact fluency instruction was 
one of several components in the interven- 
tion, and it is difficult to judge the impact 
of the fact fluency component alone. 
However, because numerous research 
teams independently produced similar 
findings, we consider this practice worthy 
of serious consideration. Although the re- 
search is limited to the elementary school 
grades, in the panel’s view, building fact 
fluency is also important for middle school 
students when used appropriately. 

Brief summary of evidence to 
support the recommendation 

The evidence demonstrates small positive 
effects on fact fluency and operations for 
the elementary grades and thus provides 
support for including fact fluency activi- 
ties as either stand-alone interventions 
or components of larger tier 2 interven- 
tions.'^' These positive effects did not, 
however, consistently reach statistical sig- 
nificance, and the findings cannot be ex- 
trapolated to areas of mathematics outside 
of fact fluency and operations. 
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RECOMMENDATION 6. INTERVENTIONS AT ALL GRADE LEVELS SHOULD DEVOTE ABOUT 10 MINUTES 



Two studies examined the effects of being 
taught mathematics facts relative to the 
effects of being taught spelling or word 
identification using similar methods. In 
both studies, the mathematics facts group 
demonstrated positive gains in fact flu- 
ency relative to the comparison group, but 
the effects were significant in only one of 
the studies. 

Another two interventions included a facts 
fluency component in combination with a 
larger tier 2 intervention. For example, 
in the Fuchs et al. (2005) study, the final 10 
minutes of a 40 minute intervention ses- 
sion were dedicated to practice with addi- 
tion and subtraction facts. In both stud- 
ies, tier 2 interventions were compared 
against typical tier 1 classroom instruc- 
tion. In each study, the effects on mathe- 
matics facts were small and not significant, 
though the effects were generally positive 
in favor of groups that received the inter- 
vention. Significant positive effects were 
detected in both studies in the domain of 
operations, and the fact fluency compo- 
nent may have been a factor in improving 
students’ operational abilities. 

Many of the studies in the evidence base 
included one or more of a variety of com- 
ponents such as teaching the relationships 
among facts, making use of a variety 
of materials such as flash cards and com- 
puter-assisted instruction, 136 and teaching 
math facts for a minimum of 10 minutes 
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per session. 337 Since these components 
were typically not independent variables 
in the studies, it is difficult to attribute any 
positive effects to the component itself. 
There is evidence, however, that strategy- 
based instruction for fact fluency (such 
as teaching the counting-on procedure) is 
superior to rote memorization. 33® 

How to carry out this 
recommendation 



1. Provide about 10 minutes per session of 
instruction to build quick retrieval of basic 
arithmetic facts. Consider using technology, 
flash cards, and other materials for extensive 
practice to facilitate automatic retrieval. 

The panel recommends providing about 10 
minutes each session for practice to help 
students become automatic in retrieving 
basic arithmetic facts, beginning in grade 

2. The goal is quick retrieval of facts using 
the digits 0 to 9 without any access to pen- 
cil and paper or manipulatives. 

Presenting facts in number families (such 
as 7 X 8 = 56, 8 X 7 = 56, 56/7 = 8, and 
56/8 = 7) shows promise for improving 
student fluency.'^** In the panel’s view, 
one advantage of this approach is that 
students simultaneously learn about the 
nature of inverse operations. 

In the opinion of the panel, cumulative re- 
view is critical if students are to maintain 
fluency and proficiency with mathematics 
facts. An efficient way to achieve this is 
to integrate previously learned facts into 
practice activities. To reduce frustration 
and provide enough extended practice so 
that retrieval becomes automatic (even for 
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al. (2006); Fuchs, Seethaler et al. (2008). 
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those who tend to have limited capacity to 
remember and retrieve abstract material), 
interventionists can individualize practice 
sets so students learn one or two new facts, 
practice several recently acquired facts, 
and review previously learned facts. If 
students are proficient in grade-level math- 
ematics facts, then the panel acknowledges 
that students might not need to practice 
each session, although periodic cumulative 
review is encouraged. 



2. For students in kindergarten through 
grade 2, explicitly teach strategies for effi- 
cient counting to improve the retrieval of 
mathematics facts. 

It is important to provide students in kin- 
dergarten through grade 2 with strategies 
for efficiently solving mathematics facts as 
a step toward automatic, fluent retrieval. 
The counting-up strategy has been used 
to increase students’ fluency in addition 
facts. This is a simple, effective strategy 
that the majority of students teach them- 
selves, sometimes as early as age But 
students with difficulties in mathematics 
tend not to develop this strategy on their 
own, even by grade 2.^^^ There is evidence 
that systematic and explicit instruction in 
this strategy is effective. 

Students can be explicitly taught to find 
the smaller number in the mathematics 
fact, put up the corresponding number of 
fingers, and count up that number of fin- 
gers from the larger number. For example, 

to solve 3 -E 5 = , the teacher identifies 

the smaller number (3) and puts up three 
fingers. The teacher simultaneously says 
and points to the larger number before 
counting three fingers, 6, 7, 8. 



140. Hasselbring, Bransford, and Coin (1988). 
Note that there was not sufficient information to 
do a WWC review. 

141. Bernie-Smith (1991); Tournaki (2003). 

142. Siegler and Jenkins (1989). 

143. Tournaki (2003). 

144. Tournaki (2003). 



Note that learning the counting-up strategy 
not only improves students’ fact fluency^^^ 
but also immerses students in the commu- 
tative property of addition. For example, 
students learn that when the larger number 

is presented second (3 -e 5 = ), they can 

rearrange the order and start counting up 
from 5. In the view of the panel, this linkage 
is an important part of intervention. After 
this type of instruction, follow-up practice 
with flash cards might help students make 
the new learning automatic. 



3. Teach students in grades 2 through 8 how 
to use their knowledge of properties, such 
as commutative, associative, and distributive 
law, to derive facts in their heads. 

Some researchers have argued that rather 
than solely relying on rote memorization 
and drill and practice, students should 
use properties of arithmetic to solve com- 
plex facts involving multiplication and di- 
vision. These researchers believe that 
by teaching the use of composition and 
decomposition, and applying the distribu- 
tive property to situations involving mul- 
tiplication, students can increasingly learn 
how to quickly (if not automatically) re- 
trieve facts. For example, to understand 
and quickly produce the seemingly difficult 

multiplication fact 13 x 7 = , students 

are reminded that 13 = 10 -e 3, something 
they should have been taught consistently 
during their elementary career. Then, since 
13 X 7 = (10 + 3) X 7 = 10 X 7 + 3 X 7, the 
fact is parsed into easier, known problems 

10 X 7 = and 3x7 = by applying 

of the distributive property. Students can 
then rely on the two simpler multiplication 
facts (which they had already acquired) to 
quickly produce an answer mentally. 

The panel recommends serious consid- 
eration of this approach as an option for 
students who struggle with acquisition of 



145. Tournaki (2003). 

146. Robinson, Menchetti, and Torgesen (2002); 
Woodward (2006). 
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facts in grades 2 through 8. When choos- 
ing an intervention curriculum, consider 
one that teaches this approach to students 
in this age range. Note, however, that the 
panel believes students should also spend 
time after instruction with extensive prac- 
tice on quick retrieval of facts through 
the use of materials such as flash cards 
or technology. 

Roadblocks and solutions 

Roadblock 6.1. Students may find fluency 
practice tedious and boring. 

Suggested Approach. Games that pro 
vide students with the opportunity to 
practice new facts and review previously 
learned facts by encouraging them to beat 
their previous high score can help the 



practice be less tedious.'"*’ Players may be 
motivated when their scores rise and the 
challenge increases. Further recommenda- 
tions for motivating students are in recom- 
mendation 8. 

Roadblock 6.2. Curricula may not include 
enough fact practice or may not have ma- 
terials that lend themselves to teaching 
strategies. 

Suggested Approach. Some contempo 
rary curricula deemphasize fact practice, 
so this is a real concern. In this case, we 
recommend using a supplemental program, 
either flash card or technology based. 



147. Fuchs, Seethaler et al. (2008). 
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Recommendation 7. 
Monitor the progress 
of students receiving 
supplemental 
instruction and 
other students 
who are at risk. 

Assess the progress of tier 2 and tier 
3 students regularly with general 
outcome measures and curriculum 
embedded measures. Also monitor 
regularly the progress of tier 1 students 
who perform just above the cutoff 
score for general outcome measures 
so they can be moved to tier 2 if they 
begin to fall behind. 

In addition, use progress monitoring 
data to determine when instructional 
changes are needed. This includes 
regrouping students who need 
continuing instructional support within 
tier 2 or tier 3, or moving students 
who have met benchmarks out of 
intervention groups and back to tier 1. 

Information about specific progress 
monitoring measures is available in 
Appendix D. A list of online resources 
is in the text below. 

Level of evidence: Low 

The panel judged the level of evidence 
supporting this recommendation to be 
low. No studies that met WWC standards 
supported this recommendation. In- 
stead, the recommendation is based on 
the panel’s expert opinion as well as con- 



148. The technical adequacy studies of mathe- 
matics progress monitoring measures were not 
experimental; the researchers typically used cor- 
relation techniques to evaluate the reliability and 
criterion validity of the measures and regression 
methods to examine sensitivity to growth. 



sideration of the standards for measure- 
ment established by a joint committee of 
national organizations. 

Brief summary of evidence to 
support the recommendation 

Although we found no studies that ad- 
dressed the use of valid measures for strug- 
gling students within an Rtl framework, 
nonexperimental studies demonstrate the 
technical adequacy of various progress 
monitoring measures. Measures for the 
primary grades typically reflect aspects of 
number sense, including strategic counting, 
numeral identification, and magnitude com- 
parisons. Studies investigating measures 
for the elementary grades focus mostly on 
the characteristics of general outcome mea- 
sures that represent grade-level mathemat- 
ics curricula in computation and in mathe- 
matics concepts and applications. widely 
used, these measures are recommended by 
the National Center for Student Progress 
Monitoring. Less evidence is available 
to support progress monitoring in middle 
school. But research teams have devel- 
oped measures focusing on math concepts 
typically taught in middle school, basic 
facts, and estimation. 



149. The American Psychological Association, 
the American Educational Research Associa- 
tion, and the National Council on Measurement 
in Education (1999). 

150. Eor example, Clarke et al. (2008); Foegen 
and Deno (2001); Fuchs et al. (1993); Fuchs, Fuchs, 
Hamlett, Thompson et al. (1994); Feh et al. (2007); 
Fembke et al. (2008). 

151. For example, Clarke et al. (2008); Fembke et 
al. (2008); Bryant, Bryant, Gersten, Scammacca, 
and Chavez (2008). 

152. Fuchs and Fuchs (1998); Fuchs et al. 
(1999). 

153. www.studentprogress.org. 

154. Foegen (2008). 

155. Helwig, Anderson, and Tindal (2002). 

156. Espin et al. (1989). 

157. Foegen and Deno (2001); Foegen (2000). 
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RECOMMENDATION 7. MONITOR THE PROGRESS OF STUDENTS RECEIVING SUPPLEMENTAL INSTRUCTION 



How to carry out this 
recommendation 



1 . Monitor the progress of tier 2, tier 3, and 
borderline tier 1 students at least once a 
month using grade-appropriate general out- 
come measures. 

General outcome measures typically take 
5 to 10 minutes to administer and should 
be used at least monthly to monitor tier 
2 and tier 3 students. General outcome 
measures use a sample of items from the 
array of concepts covered over one year to 
assess student progress. They provide a 
broad perspective on student proficiency 
in mathematics. They target concepts such 
as magnitude comparison, counting ability, 
and knowledge of place value for students 
in kindergarten and grade 1, and increas- 
ingly complex aspects of place value and 
proficiency with operations for students 
in grades 2 through 6. Examining student 
performance on these measures allows 
teachers to determine whether students are 
integrating and generalizing the concepts, 
skills, and strategies they are learning in the 
core curriculum and the intervention.^^® 

In addition to monitoring the progress of 
tier 1 and tier 2 students, the panel recom- 
mends monitoring the progress of border- 
line tier 1 students with general outcome 
measures on a monthly basis. Since these 
students scored just above the cut score, 
they were not selected for supplemental 
instruction. The panel suggests using one 
standard error of measurement (a statistic 
available in the technical information for 
the measures) above the cut score to de- 
fine the range of scores for borderline stu- 
dents. Using this approach, teachers can 
continue to monitor the progress of stu- 
dents whose scores fell just above the cut 
score and determine whether they should 
receive supplemental instruction. 



158. Fuchs, Fuchs, and Zumeta (2008). 



Choose progress monitoring measures 
with evidence supporting their reliability, 
validity, and ability to identify growth. 
This will require input from individu- 
als with expertise in these areas, typi- 
cally school psychologists or members of 
district research departments. Consider 
whether the measure produces consistent 
results (reliability) and provides informa- 
tion that correlates with other measures 
of mathematics achievement (criterion 
validity). Ability to identify growth helps 
interventionists ensure that students are 
learning and making progress toward an 
annual goal through the full array of ser- 
vices they are receiving. 

In some cases, general outcome measures 
may also be used for screening, as de- 
scribed in recommendation 1. Resources 
that teachers can turn to for identifying 
appropriate measures include the Na- 
tional Center on Student Progress Moni- 
toring’s review of available tools (http:// 
www.studentprogress.org/) and the Re- 
search Institute on Progress Monitoring 
(http://www.progressmonitoring.org/). 



2. Use curriculum-embedded assessments 
in interventions to determine whether stu- 
dents are learning from the intervention. 
These measures can be used as often as 
every day™ or as infrequently as once every 
other week.'®° 

Many tier 2 and tier 3 intervention pro- 
grams (commercially developed, re- 
searcher developed, or district developed) 
include curriculum-embedded assess- 
ments (sometimes called unit tests, mas- 
tery tests, or daily probes). The results of 
these assessments can be used to deter- 
mine which concepts need to be reviewed, 
which need to be re-taught, and which have 
been mastered. Curriculum-embedded as- 
sessments are often administered daily 



159. Bryant, Bryant, Gersten, Scammacca, and 
Chavez (2008). 

160. Jitendra (2007). 
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for students in kindergarten^®^ and grade 
1 and biweekly for students in grades 2 
through These assessments usually 
do not possess the same high technical 
characteristics of the general outcome 
measures. Curriculum-embedded assess- 
ments often result in very useful informa- 
tion for interventionists because they can 
detect changes in student performance in 
the concepts and skills being taught at the 
time. Interventionists need to be cautious 
about assuming that mastery of individ- 
ual skills and concepts will translate into 
improvements in overall proficiency. As a 
result, the panel recommends using both 
general outcome measures and curricu- 
lum-embedded assessments for students 
receiving interventions. 

If the intervention program does not in- 
clude curriculum-embedded assessments, 
use efficient, reliable, and valid screen- 
ing measures, which can also be used as 
progress monitoring measures (see recom- 
mendation 1). 



3. Use progress monitoring data to regroup 
students when necessary. 

Since student skill levels change over 
time and in varying degrees, the panel 



161. For example, one tier 2 intervention pro- 
gram for 1st and 2nd grade students reported by 
Bryant, Bryant, Gersten, Scammacca, and Chavez 
(2008) included daily activity-level progress 
monitoring that consisted of four oral or written 
problems drawn from the content focus for that 
day. Teachers were instructed that a majority of 
the students in the group had to complete at least 
three of the four problems correctly to consider 
the daily lesson successful. 

162. A parallel example in grades 3 and beyond 
can be found in Jitendra’s Solving math word 
problems instructional materials on teaching 
word problems (2007). There are many other ex- 
amples in available commercial programs. 



recommends using progress monitoring 
data to regroup students within tiers so that 
the small groups used in tier 2 interventions 
are as homogeneous as possible. If a stu- 
dent does not fit into any of the intervention 
groups from his or her class, consider put- 
ting the child in an intervention group from 
another class if the schedule permits. 

Roadblocks and solutions 

Roadblock 7.1. Students within classes are 
at very different levels. This can make it dif- 
ficult to group students into appropriate tier 
2 and tier 3 intervention groups. 

Suggested Approach. If students within 
a class are at such diverse levels that ap- 
propriate tier 2 and tier 3 intervention 
groups cannot be made, consider grouping 
students across classes. This will facilitate 
clustering students with similar needs. For 
example, teachers of upper elementary 
students may find that students who have 
not yet mastered basic concepts in a par- 
ticular area (fractions) are spread across 
several classrooms. Putting these students 
in a single tier 2 intervention group would 
be the most efficient means of meeting 
their needs, rather than trying to provide 
one or two students in each class with 
services duplicated across classrooms. In 
such a case, a math specialist, parapro- 
fessional, or other school personnel who 
have received training can conduct the 
intervention. 

Roadblock 7.2. There is insufficient 
time for teachers to implement progress 
monitoring. 

Suggested Approach. If teachers are too 
busy to assess student progress with moni- 
toring measures, consider training parapro- 
fessionals or other school staff to do so. 
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Recommendation 8. 
Include motivational 
strategies in tier 2 and 
tier 3 interventions. 

Adults can sometimes forget how 
challenging so-called “basic” arithmetic 
is for students in tier 2 and tier 3 
interventions. Many of these students 
have had experiences of failure and 
frustration with mathematics by the 
time they receive an intervention. They 
may also have a particularly difficult 
time storing and easily retrieving 
information in their memories.’®^ 
Therefore, it seems particularly 
important to provide additional 
motivation for these students.'®"* 

Praising students for their effort 
and for being engaged as they work 
through mathematics problems is a 
powerful motivational tool that can 
be effective in increasing students’ 
academic achievement.'®® Tier 2 and 
tier 3 interventions should include 
components that promote student 
effort {engagement-contingent 
rewards), persistence {completion- 
contingent rewards), and achievement 
{performance-contingent rewards). 
These components can include praise 
and rewards. Even a well-designed 
intervention curriculum may falter 
without such behavioral supports. 



163. Geary (2003). 

164. The scope of this practice guide limited the 
motivational strategies reviewed to strategies 
used in studies of students struggling with math- 
ematics. For a wider review of effective motiva- 
tional strategies used in classrooms, see Epstein 
et al. (2008) and Halpern et al. (2007). 

165. Schunk and Cox (1986); Fuchs et al. (2005). 
Fuchs, Fuchs, Craddock et al. (2008). 



Level of evidence: Low 

The panel judged the level of evidence 
supporting this recommendation to be 
low. This recommendation is based on 
the professional opinion of the panel, and 
on nine studies that met WWC standards 
or met standards with reservations that 
included motivational strategies in the 
intervention. Although one of these 
studies demonstrated that praising strug- 
gling students for their effort significantly 
improved their ability to solve subtraction 
problems with regrouping, other stud- 
ies included a motivational component as 
one of several components of the inter- 
vention. In the opinion of the panel, these 
studies did not show that a motivational 
component is essential but suggest that it 
may be useful for improving mathematics 
achievement.'®® 

Brief summary of evidence to 
support the recommendation 

One study that met WWC standards exam- 
ined the effects of a motivational component 
by comparing the performance of students 
who received praise for their effort dur- 
ing subtraction instruction with those who 
did not receive praise.'®® This study found 
significant positive effects on student sub- 
traction scores in favor of providing effort 
feedback."'® Although this study provides 
some evidence of the effectiveness of a mo- 
tivational strategy, it is the only study that 



166. Fuchs et al. (2005); Fuchs, Fuchs, Cradock et al. 
(2008); Schunk and Cox (1986); Fuchs, Seethaler et al. 
(2008); Heller and Fantuzzo (1993); Artus and Dyrek 
(1989); Fuchs, Fuchs et al. (2003b); Fuchs, Fuchs, 
Hamlett, Phillips et al. (1994); Fuchs, Fuchs, Finelli 
et al. (2006). 

167. Schunk and Cox (1986). 

168. There is an extensive literature on motiva- 
tional strategies outside the scope of this prac- 
tice guide. For more information on motivational 
strategies see Epstein et al. (2008) and Halpern 
et al. (2007). 

169. Schunk and Cox (1986). 

170. Schunk and Cox (1986). 



( 44 ) 
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explicitly tested the effects of motivational 
strategies on mathematics outcomes. 

In two studies, students received points 
for engagement and attentiveness, and 
in three studies, students were provided 
with prizes as tangible reinforcers for ac- 
curate mathematics problem-solving. 
However, in each of these studies, it was 
not possible to isolate the effects of rein- 
forcing attentiveness and accuracy. For 
example, in two of the studies, students 
in tier 2 tutoring earned prizes for accura- 
cy.^^^ Although in both studies, the tier 2 
intervention group demonstrated substan- 
tively important positive and sometimes 
significant gains on a variety of mathemat- 
ics measures relative to the students who 
remained in tier 1, it is not possible to iso- 
late the effects of the reinforcers from the 
provision of tier 2 tutoring. Another study 
examined the impact of parental involve- 
ment on students’ mathematics achieve- 
ment and found statistically significant 
positive effects on operations and general 
math achievement. However, because 
the parental involvement component was 
multifaceted, it is not possible to attribute 
the positive effects to rewards alone. 

Five studies in the evidence base included 
interventions in which students graphed 
their progress and in some cases set goals 
for improvement on future assessments. 
One experimental study examined the 
effects of student graphing and goal set- 
ting as an independent variable and found 
substantively important positive effects 
on measures of word problems in favor 



171. Fuchs et al. (2005); Fuchs, Fuchs, Craddock 
et al. (2008). 

172. Fuchs et al. (2005); Fuchs, Seethaler et al. 
(2008); Fuchs, Fuchs, Craddock et al. (2008). 

173. Fuchs et al. (2005); Fuchs, Seethaler et al. 
(2008). 

174. Heller and Fantuzzo (1993). 

175. Artus and Dyrek (1989); Fuchs, Seethaler et 
al. (2008); Fuchs et al. (2003b); Fuchs, Fuchs, Ham- 
lett, Phillips et al. (1994); Fuchs, Fuchs, Finelli 
et al. (2006). 



of students who graphed and set goals. 
The other four studies did not isolate the 
effects of graphing progress. Because 
this recommendation is based primarily 
on the opinion of the panel, the level of 
evidence is identified as low. 

How to carry out this 
recommendation 



1. Reinforce or praise students for their effort 
and for attending to and being engaged in 
the lesson. 

Verbally praise students for their effort^^® 
and for listening carefully and following 
the lesson in a systematic fashion {engage- 
ment-contingent rewards).^^'^ The panel 
believes that praise should be immediate 
and specific to highlight student effort 
and engagement. But we also believe that 
it is ineffective to offer generic and empty 
praise (“good job!” or “keep up the good 
work!”) that is not related to actual effort. 
Instead, praise is most effective when it 
points to specific progress that students 
are making and recognizes students’ ac- 
tual effort.^®° Systematically praising stu- 
dents for their effort and engagement may 
encourage them to remain focused on the 
completion of their work. 



2. Consider rewarding student accom- 
plishments. 

Consider using rewards to acknowledge 
completion of math tasks {completion- 
contingent rewards) and accurate work 
{performance-contingent rewards). This 
can be done by applauding or verbally 



176. Fuchs et al. (2003b). 

177. Artus and Dyrek (1989); Fuchs, Seethaler et 
al. (2008); Fuchs, Fuchs, Hamlett, Phillips et al. 
(1994); Fuchs, Fuchs, Finelli et al. (2006). 

178. Schunkand Cox (1986). 

179. For example, Fuchs et al. (2005); Fuchs, 
Fuchs, Craddock et al. (2008). 

180. See Bangert-Drowns et al. (1991) and Halpern 
et al. (2007) for a review. 
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praising students for actual accomplish- 
ments, such as finishing assignments, im- 
proving their score from 70 percent to 80 
percent correct, or giving students points 
or tokens each time they answer a problem 
correctly, which they can use to “buy” tangi- 
ble rewards at a later time.^®^ Again, praise 
should be specific rather than generic. 
Consider notifying the student’s parents 
to inform them of their child’s successes in 
mathematics by phone or email or in a note 
sent home with the student.^®^ Remember 
that parents of these students are likely to 
receive notification of problems rather than 
successes, and some evidence suggests that 
this specific positive attention might sup- 
port achievement growth.^®'* 



3. Allow students to chart their progress and 
to set goals for improvement. 

Several of the interventions in the evi- 
dence base for this practice guide had stu- 
dents graph their progress on charts^®^ and 
set goals for improving their assessment 
scores.^®® For example, students might 
graph their scores on a chart showing a se- 
ries of thermometers, one for each session 
of the intervention.^®^ At the beginning of 
each session, students can examine their 
charts and set a goal to beat their previ- 
ous score or to receive the maximum score. 
This type of goal setting is believed to help 
students develop self-regulated learning 
because students take independent respon- 
sibility for setting and achieving goals. ^®® 



181. For example, Fuchs et al. (2005); Fuchs, 
Seethaler et al. (2008); Fuchs, Fuchs, Craddock 
et al. (2008). 

182. Halpern et al. (2007). 

183. For example, Heller and Fantuzzo (1993). 

184. For example, Heller and Fantuzzo (1993). 

185. Artus and Dyrek (1989); Fuchs, Fuchs, Ham- 
lett, Phillips et al. (1994); Fuchs, Fuchs, Finelll et 
al. (2006); Fuchs, Seethaler et al. (2008). 

186. Fuchs et al. (2003b). 

187. See the procedure in Fuchs et al. (2003b). 

188. Fuchs et al. (1997). 



Roadblocks and suggested 
approaches 

Roadblock 8.1. Rewards can reduce genu- 
ine interest in mathematics by directing stu- 
dent attention to gathering rewards rather 
than learning math. 

Suggested Approach. It is important to 
inform interventionists that research in 
other content areas has demonstrated that 
rewards and praise increase the likelihood 
of students’ academic success without 
diminishing their interest in learning. i®^ 
Given the frequent history of failure for 
many of these students, at least in the ele- 
mentary grades, we suggest using rewards 
and praise to encourage effort, engage- 
ment, and achievement. As students learn 
and succeed more often in mathematics, 
interventionists can gradually fade the 
use of rewards because student success 
will become an intrinsic reward. The WWC 
Reducing Behavior Problems in the Elemen- 
tary School Classroom Practice Cuide^^° is 
a good reference for more information on 
the use of rewards and praise. 

Roadblock 8.2. It is difficult to deter- 
mine appropriate rewards for individual 
students. 

Suggested Approach. Consider each stu- 
dent’s interests before choosing an appro- 
priate reward. Also consider using oppor- 
tunities to engage in activities students are 
interested in as rewards to reinforce effort, 
engagement, and accurate work. Parents 
may also have ideas for rewards that will 
help motivate their children. Schools can 
engage parents in rewarding students and 
coordinate efforts to reward children at 
home as well.^®^ 



189. Epstein et al. (2008). 

190. Epstein et al. (2008). 

191. For example, Heller and Fantuzzo (1993). 
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Roadblock 8.3. Providing feedback and 
rewarding achievement detracts from class- 
room instructional time. It is difficult to fit 
it into the classroom schedule. 

Suggested Approach. Verbally prais 
ing students for their effort individually 
and their engagement in small group les- 
sons requires very little time. Awarding 
points or tokens for correct responses 



can be done when the teacher grades the 
student’s work. To reduce the amount of 
time it takes for students to “buy” prizes 
with accumulated points or tokens, ask 
students to choose which prize they want 
before they “buy” it. The prizes can then be 
distributed quickly at the end of the day, 
so that students are not distracted by the 
items throughout the school day. 
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Glossary of terms as 
used in this report 



“minimum addend strategy” or the strategy 
of “counting on from larger”). 



The associative property of addition 

states that (A + B) + C = A + (B + C) for all 
numbers A, B, C. This property allows for 
flexibility in calculating sums. For exam- 
ple, to calculate 85 + 97 + 3, we do not have 
to add 85 and 97 first but may instead cal- 
culate the easier sum 85 -i- (97 -i- 3), which 
is 85 -I- 100, which equals 185. The associa- 
tive property is also used in deriving basic 
addition facts from other basic facts and 
therefore helps with learning these basic 
facts. For example, to add 8 -i- 5, a child can 
think of breaking the 5 into 2-1-3, combin- 
ing the 2 with the 8 to make a 10, and then 
adding on the 3 to make 13. In an equation, 
this can be recorded as: 8 -i- 5 = 8 -i- (2 -i- 3) = 
(8 + 2) + 3 = 10 + 3 = 13. 

The associative property of multiplica- 
tion states that (A x B) x C = A x (B x C) for all 
numbers A, B, C. This property allows for 
flexibility in calculating products. For exam- 
ple, to calculate 87 x 25 x 4, we do not have 
to multiply 87 and 25 first but may instead 
calculate the easier product 87 x (25 x 4), 
which is 87 x 100, which equals 8,700. The 
associative property is also used in deriv- 
ing basic multiplication facts from other 
basic facts and therefore helps with learn- 
ing these facts. For example, to calculate 
7 X 6, a child who already knows 7x3 = 21 
can double the 21 to calculate that 7 x 6 = 42. 
In an equation, this can be recorded as: 
7 X 6 = 7 X (3 X 2) = (7 X 3) X 2 = 21 X 2 = 42. 

The commutative property of addition 

states that A + B = B + A for all numbers 
A, B. This property allows for flexibility in 
calculating sums and helps lighten the load 
of learning the basic addition facts. For ex- 
ample, to add 2 + 9, a child might want to 
count on 9 from 2 by counting “3, 4, 5, 6, . . . 
11,” which is cumbersome. By applying the 
commutative property, a child can instead 
add 9 + 2 by counting on 2 from 9, by say- 
ing “10, 11,” which is much easier (this is the 



The commutative property of multipli- 
cation states that A x B = B x A for all num- 
bers A, B. This property allows for flex- 
ibility in calculating products and helps 
lighten the load of learning the basic mul- 
tiplication facts. As shown in example 9, 
once a child has learned the multiplication 
fact 3x7 = 21, the child will also know the 
fact 7 X 3 = 21 if the child understands the 
commutative property. 



Example 9: Commutative property 
of multiplication 




3 groups of 7 3x7 dots 








7 groups of 3 



7x3 dots 



Both use the same array, so the total 
is the same: 3 x 7 = 7 x 3 



Concurrent validity refers to the correla- 
tion between the assessment that is being 
investigated and a similar assessment 
when the assessments are completed at 
the same point in time. Correlation coef- 
ficients range from -1 to 1. A correlation 
coefficient close to 1 indicates a strong 
overlap between the assessments. 

Counting up/on is a strategy that young 
children can use to solve addition (and 
subtraction) problems. To calculate 8 + 3 
by counting on, the child starts with 8 and 
then “counts on” 3 more, saying “9, 10, 11.” 
The child may use fingers in order to de- 
termine when to stop counting. 
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Counting up/on from a larger addend 

is a strategy in which children apply count- 
ing on to solve an addition problem, but 
first apply the commutative property, if 
necessary, in order to count on from the 
larger number. For example, to calculate 
3 + 9, a child first changes the problem to 

9 + 3 and then counts on 3 from 9. 

Counting up/on to solve unknown ad- 
dend and subtraction problems. Count- 
ing on can be used to solve unknown ad- 
dend problems such as 11 + ? = 15. To solve 
this problem, the child can count on from 
11 to 15, counting 12, 13, 14, 15 and raising 
one finger for each number. Since 4 fingers 
were raised, 11 + 4 = 15. To solve a sub- 
traction problem such as 15 minus 11 = ? 
using counting on, the child must first un- 
derstand that the problem can be reformu- 
lated as 11 + ? = 15. Then the child can use 
counting on as described previously. Note 
too that solving this subtraction problem 
using the counting on strategy is much 
easier than counting down 11. Note too 
that the reformulation of a subtraction 
problem as an unknown addend problem 
is important in its own right because it 
connects subtraction with addition. 

In mathematics assessment, criterion- 
related validity means that student scores 
on an assessment should correspond to 
their scores or performance on other in- 
dicators of mathematics competence, such 
as teacher ratings, course grades, or stan- 
dardized test scores. 

Derived fact strategies in addition and 
subtraction are strategies in which chil- 
dren use addition facts they already know 
to find related facts. Especially important 
among derived fact strategies are the make- 
a-10 methods because they emphasize base 

10 structure. As shown in example 10, to 
add 8 + 5, a child can think of breaking 
the 5 into 2 + 3, combining the 2 with the 
8 to make a 10, and then adding on the 3 
to make 13 (see the associative property of 
addition). Note that to use this make-a-10 



strategy, children must know the “10 part- 
ner” (number that can be added to make 
10) for each number from 1 to 9 and must 
also know how to break each number into 
a sum of two (positive whole) numbers in 
all possible ways. Furthermore, the child 
must understand all the “teen” numbers 
(from 11 to 19) as a 10 and some ones (for 
example, 15 is 10 and 5 ones). 



Example 10: Make-a-10 strategy 



Li", 




••••• 

8+5 


••••• •••OO 


ooo 


= 10 + 


3=13 



Derived fact strategies in multiplica- 
tion and division are strategies in which 
children use multiplication facts they al- 
ready know to find related facts. For exam- 
ple 5 x 8 is half of 10 X 8, and similarly for 
all the “5 times” facts (these are examples 
of applying the associative property). Also, 
9 X 8 is 8 less than 10 x 8 , and similarly for 
all the “9 times” facts (these are examples 
of applying the distributive property). To 
calculate 4 x 7, we can double the double 
of 7, that is, the double of 7 is 14 and the 
double of 14 is 28, which is 4 times 7. All 
the “4 times” facts can be derived by dou- 
bling the double (these are examples of 
applying the associative property). 

The distributive property relates addi 
tion and multiplication. It states that A x 
(B + C) = (A X B) + (A X C) for all numbers 
A, B, C. This property allows for flexibil- 
ity in calculating products. For example, 
to calculate 7 x 13, we can break 13 apart 
by place value as 10 + 3 and calculate 
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7 X 10 = 70 and 7 x 3 = 21 and add these 
two results to find 7x13 = 91. In an equa- 
tion, this can be recorded as: 7 x 13 = 7 x 
(10 + 3) = (7 X 10) + (7 X 3) = 70 + 21 = 91. 
This strategy of breaking numbers apart 
by place value and applying the distribu- 
tive property is the basis for the common 
method of longhand multiplication. The 
distributive property is also used in deriv- 
ing basic multiplication facts from other 
basic facts and therefore helps in learning 
these facts. As shown in example 11, to 
calculate 6 x 7, a child who already knows 
6 X 5 = 30 and 6 x 2 = 12 can add the 30 
and 12 to calculate that 6 x 7 = 30 -f 12 = 42. 
In an equation, this can be recorded as: 
6x7 = 6x(5 + 2) = (6x5)+(6x2) = 30+ 12 = 42. 



needing help in mathematics) divided by 
the sum of this value and the number of 
false negatives, while specificity is equal 
to the number of true negatives divided 
by the sum of this value and the number 
of false positives (students misidentified 
during screening). 

A general outcome measure refers to a 
measure of specific proficiencies within 
a broader academic domain. These profi- 
ciencies are related to broader outcomes. 
For example, a measure of oral reading flu- 
ency serves as a general outcome measure 
of performance in the area of reading. The 
measures can be used to monitor student 
progress over time. 



Example 11: Distributive property 



•••••oo 

•••••oo 

•••••oo 

•••••oo 

•••••oo 

•••••oo 



We can break 
6 rows of 7 into 
6 rows of 5 and 
6 rows of 2. 



6x7 = 6x5 -F 6x2 

=30 + 12 = 42 



Efficiency is how quickly the universal 
screening measure can be administered, 
scored, and analyzed for all the students 
tested. 

False positives and false negatives are 

technical terms used to describe the mis- 
identification of students. The numbers 
of false positives and false negatives are 
related to sensitivity and specificity. As 
depicted in table 3 (p. 16) of this guide, 
sensitivity is equal to the number of true 
positives (students properly identified as 



Interventionist refers to the person 
teaching the intervention. The interven- 
tionist might be a classroom teacher, in- 
structional assistant, or other certified 
school personnel. 

The magnitude of a quantity or number is 
its size, so a magnitude comparison is a 

comparison of size. The term magnitude is 
generally used when considering size in an 
approximate sense. In this case, we often 
describe the size of a quantity or number 
very roughly by its order of magnitude, 
which is the power of ten (namely 1, 10, 

100, 1000 or 0.1, 0.01, 0.001 ) that 

the quantity or number is closest to. 

Number composition and number de- 
composition are not formal mathematical 
terms but are used to describe putting num- 
bers together, as in putting 2 and 3 together 
to make 5, and breaking numbers apart, as in 
breaking 5 into 2 and 3. For young children, 
a visual representation like the one shown 
on the next page is often used before intro- 
ducing the traditional mathematical notation 
2 -F 3 = 5 and 5 = 2 -f 3 for number composi- 
tion and decomposition. 
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Example 12: Number decomposition 




A number line is a line on which locations 
for 0 and 1 have been chosen (1 is to the 
right of 0, traditionally). Using the distance 
between 0 and 1 as a unit, a positive real 
number, N, is located N units to the right of 
0 and a negative real number, -N, is located 
N units to the left of 0. In this way, every 
real number has a location on the number 
line and every location on the number line 
corresponds to a real number. 

A number path is an informal precursor 
to a number line. It is a path of consecu- 
tively numbered “steps,” such as the paths 
found on many children’s board games 
along which game pieces are moved. De- 
termining locations on number paths only 
requires counting, whereas determining 
locations on number lines requires the no- 
tion of distance. 



do in mathematics a year or even two or 
three years later. 

Response to Intervention (Rtl) is an 
early detection, prevention, and support 
system in education that identifies strug- 
gling students and assists them before 
they fall behind. 

Sensitivity indicates how accurately a 
screening measure predicts which stu- 
dents are at risk. Sensitivity is calculated 
by determining the number of students 
who end up having difficulty in mathemat- 
ics and then examining the percentage of 
those students predicted to be at risk on 
the screening measure. A screening mea- 
sure with high sensitivity would have a 
high degree of accuracy. In general, sen- 
sitivity and specificity are related (as one 
increases the other usually decreases). 

Specificity indicates how accurately a 
screening measure predicts which stu- 
dents are not at risk. Specificity is cal- 
culated by determining the number of 
students who do not have a deficit in math- 
ematics and then examining the percent- 
age of those students predicted to not be at 
risk on the screening measure. A screening 
measure with high specificity would have 
a high degree of accuracy. In general, sen- 
sitivity and specificity are related (as one 
increases the other usually decreases). 



Reliability refers to the degree to which 
an assessment yields consistency over 
time (how likely are scores to be similar if 
students take the test a week or so later?) 
and across testers (do scores change when 
different individuals administer the test?). 
Alternate form reliability tells us the ex- 
tent to which an educator can expect sim- 
ilar results across comparable forms or 
versions of an assessment. 

Predictive validity is the extent to which 
a test can predict how well students will 



Strip diagrams (also called model dia- 
grams and bar diagrams) are drawings of 
narrow rectangles that show relationships 
among quantities. 

A validity coefficient serves as an index 
of the relation between two measures and 
can range from -1.0 to 1.0, with a coeffi- 
cient of .0 meaning there is no relation 
between the two scores and increasing 
positive scores indicating a stronger posi- 
tive relation. 
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the Institute of 
Education Sciences 

What is a practice guide? 

The health care professions have em- 
braced a mechanism for assembling and 
communicating evidence-based advice to 
practitioners about care for specific clini- 
cal conditions. Variously called practice 
guidelines, treatment protocols, critical 
pathways, best practice guides, or simply 
practice guides, these documents are sys- 
tematically developed recommendations 
about the course of care for frequently en- 
countered problems, ranging from physi- 
cal conditions, such as foot ulcers, to psy- 
chosocial conditions, such as adolescent 
development.^®^ 

Practice guides are similar to the prod- 
ucts of typical expert consensus panels 
in reflecting the views of those serving 
on the panel and the social decisions that 
come into play as the positions of individ- 
ual panel members are forged into state- 
ments that all panel members are willing 
to endorse. Practice guides, however, are 
generated under three constraints that do 
not typically apply to consensus panels. 
The first is that a practice guide consists 
of a list of discrete recommendations that 
are actionable. The second is that those 
recommendations taken together are in- 
tended to be a coherent approach to a 
multifaceted problem. The third, which is 
most important, is that each recommen- 
dation is explicitly connected to the level 
of evidence supporting it, with the level 
represented by a grade (strong, moder- 
ate, or low). 

The levels of evidence, or grades, are 
usually constructed around the value of 
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particular types of studies for drawing 
causal conclusions about what works. 
Thus, one typically finds that a strong 
level of evidence is drawn from a body of 
randomized controlled trials, the moder- 
ate level from well-designed studies that 
do not involve randomization, and the 
low level from the opinions of respected 
authorities (see table 1, p. 2). Levels of evi- 
dence also can be constructed around the 
value of particular types of studies for 
other goals, such as the reliability and va- 
lidity of assessments. 

Practice guides also can be distinguished 
from systematic reviews or meta-analyses 
such as What Works Clearinghouse (WWC) 
intervention reviews or statistical meta- 
analyses, which employ statistical meth- 
ods to summarize the results of studies 
obtained from a rule-based search of the 
literature. Authors of practice guides sel- 
dom conduct the types of systematic lit- 
erature searches that are the backbone of 
a meta-analysis, although they take ad- 
vantage of such work when it is already 
published. Instead, authors use their ex- 
pertise to identify the most important 
research with respect to their recommen- 
dations, augmented by a search of recent 
publications to ensure that the research 
citations are up-to-date. Furthermore, the 
characterization of the quality and direc- 
tion of the evidence underlying a recom- 
mendation in a practice guide relies less 
on a tight set of rules and statistical algo- 
rithms and more on the judgment of the 
authors than would be the case in a high- 
quality meta-analysis. Another distinction 
is that a practice guide, because it aims for 
a comprehensive and coherent approach, 
operates with more numerous and more 
contextualized statements of what works 
than does a typical meta-analysis. 

Thus, practice guides sit somewhere be- 
tween consensus reports and meta-analyses 
in the degree to which systematic processes 
are used for locating relevant research and 
characterizing its meaning. Practice guides 
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are more like consensus panel reports than 
meta-analyses in the breadth and complex- 
ity of the topic that is addressed. Practice 
guides are different from both consensus 
reports and meta-analyses in providing 
advice at the level of specific action steps 
along a pathway that represents a more-or- 
less coherent and comprehensive approach 
to a multifaceted problem. 

Practice guides in education at the 
Institute of Education Sciences 

lES publishes practice guides in educa- 
tion to bring the best available evidence 
and expertise to bear on the types of sys- 
temic challenges that cannot currently be 
addressed by single interventions or pro- 
grams. Although lES has taken advantage 
of the history of practice guides in health 
care to provide models of how to proceed 
in education, education is different from 
health care in ways that may require that 
practice guides in education have some- 
what different designs. Even within health 
care, where practice guides now number 
in the thousands, there is no single tem- 
plate in use. Rather, one finds descriptions 
of general design features that permit 
substantial variation in the realization 
of practice guides across subspecialties 
and panels of experts. Accordingly, the 
templates for lES practice guides may vary 
across practice guides and change over 
time and with experience. 

The steps involved in producing an lES- 
sponsored practice guide are first to select 
a topic, which is informed by formal sur- 
veys of practitioners and requests. Next, a 
panel chair is recruited who has a national 
reputation and up-to-date expertise in the 
topic. Third, the chair, working in collabo- 
ration with lES, selects a small number of 
panelists to co-author the practice guide. 
These are people the chair believes can 
work well together and have the requisite 
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expertise to be a convincing source of rec- 
ommendations. lES recommends that at 
least one of the panelists be a practitioner 
with experience relevant to the topic being 
addressed. The chair and the panelists are 
provided a general template for a practice 
guide along the lines of the information 
provided in this appendix. They are also 
provided with examples of practice guides. 
The practice guide panel works under a 
short deadline of six to nine months to pro- 
duce a draft document. The expert panel 
members interact with and receive feed- 
back from staff at lES during the develop- 
ment of the practice guide, but they under- 
stand that they are the authors and, thus, 
responsible for the final product. 

One unique feature of lES-sponsored prac- 
tice guides is that they are subjected to 
rigorous external peer review through the 
same office that is responsible for inde- 
pendent review of other lES publications. 
A critical task of the peer reviewers of a 
practice guide is to determine whether 
the evidence cited in support of particular 
recommendations is up-to-date and that 
studies of similar or better quality that 
point in a different direction have not been 
ignored. Peer reviewers also are asked to 
evaluate whether the evidence grade as- 
signed to particular recommendations by 
the practice guide authors is appropriate. 
A practice guide is revised as necessary to 
meet the concerns of external peer reviews 
and gain the approval of the standards and 
review staff at lES. The process of external 
peer review is carried out independent of 
the office and staff within lES that insti- 
gated the practice guide. 

Because practice guides depend on the 
expertise of their authors and their group 
decisionmaking, the content of a practice 
guide is not and should not be viewed as a 
set of recommendations that in every case 
depends on and flows inevitably from sci- 
entific research. It is not only possible but 
also likely that two teams of recognized 
experts working independently to produce 
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a practice guide on the same topic would 
generate products that differ in important 
respects. Thus, consumers of practice 
guides need to understand that they are, 
in effect, getting the advice of consultants. 
These consultants should, on average, pro- 
vide substantially better advice than an 



individual school district might obtain on 
its own because the authors are national 
authorities who have to reach agreement 
among themselves, justify their recom- 
mendations in terms of supporting evi- 
dence, and undergo rigorous independent 
peer review of their product. 

Institute of Education Sciences 
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Appendix C. 

Disclosure of potential 
conflicts of interest 

Practice guide panels are composed of in- 
dividuals who are nationally recognized 
experts on the topics about which they 
are rendering recommendations. The In- 
stitute of Education Sciences (lES) expects 
that such experts will be involved profes- 
sionally in a variety of matters that relate 
to their work as a panel. Panel members 
are asked to disclose their professional 
involvements and to institute deliberative 
processes that encourage critical exami- 
nation of the views of panel members as 
they relate to the content of the practice 
guide. The potential influence of panel 
members’ professional engagements is 
further muted by the requirement that 
they ground their recommendations in evi- 
dence documented in the practice guide. 
In addition, the practice guide undergoes 
independent external peer review prior 
to publication, with particular focus on 
whether the evidence related to the rec- 
ommendations in the practice guide has 
been appropriately presented. 

The professional engagements reported 
by each panel member that appear most 
closely associated with the panel recom- 
mendations are noted below. 

Russell Gersten has written articles on 
issues related to assessment and screen- 
ing of young children with potential dif- 
ficulties in learning mathematics and is 
currently revising a manuscript on this 
topic for the scholarly journal. Exceptional 
Children. However, there is no fiscal re- 
ward for this or other publications on the 
topic. He is a royalty author for what may 
become the Texas or national edition of 
the forthcoming (2010/11) Houghton Mif- 
flin reading series. Journeys. At the time 
of publication, Houghton Mifflin has not 
determined whether they will ever release 



this series. Dr. Gersten provided guidance 
on the product as it relates to struggling 
and English language learner students. 
This topic is not covered in this prac- 
tice guide. The panel never discussed the 
Houghton Mifflin series. Russell Gersten 
has no financial stake in any program or 
practice mentioned in the practice guide. 

Sybilla Beckmann receives royalties on 
her textbook. Mathematics for Elementary 
Teachers, published by Addison-Wesley, a 
division of Pearson Education. This text- 
book, used in college mathematics courses 
for prospective elementary teachers, is 
not within the scope of the review of this 
practice guide. 

Ben Clarke has developed and authored 
journal articles about early numeracy 
measures that are referenced in the Rtl- 
Mathematics practice guide. Dr. Clarke 
does not have a current financial stake 
in any company or publishing of the 
measures. 

Anne Foegen conducts research and has 
developed progress monitoring assessments 
that are referred to in the Rtl-Mathematics 
practice guide. Currently, she has no finan- 
cial interests in these products, which are 
not available commercially. The algebra 
measures are disseminated through Iowa 
State University on a fee-based schedule 
to cover the costs of personnel for training 
and materials (not for profit). Dr. Eoegen 
also has published papers that describe 
the measures and received a grant that is 
supporting a portion of the research. She 
occasionally does private consulting related 
to research on use of the mathematics prog- 
ress monitoring measures. 

Jon R. Star consults for a company owned 
by Scholastic, which produces mathemat- 
ics educational software. Scholastic may 
produce other curricula related to mathe- 
matics, but the panel makes no recommen- 
dations for selecting specific curricula. 
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60 



Appendix D. 

Technical information 
on the studies 

Recommendation 1. 

Screen all students to identify 
those at risk for potential 
mathematics difficulties and 
provide interventions to students 
identified as at risk. 

Level of evidence: Moderate 

The panel examined reviews of the tech- 
nical adequacy of screening measures for 
students identified as at risk when making 
this recommendation. The panel rated the 
level of evidence for recommendation 1 as 
moderate because several reviews were 
available for evidence on screening mea- 
sures for younger students. However, 
there was less evidence available on these 
measures for older students. The panel 
relied on the standards of the American 
Psychological Association, the American 
Educational Research Association, and the 
National Council on Measurement in Edu- 
cation^^^ for valid screening instruments 
along with expert judgment to evaluate 
the quality of the individual studies and 
to determine the overall level of evidence 
for this recommendation. 

Relevant studies were drawn from recent 
comprehensive literature reviews and re- 
ports^®^ as well as literature searches of 
databases using key terms (such as “for- 
mative assessment”). Journal articles sum- 
marizing research studies on screening 
in mathematics,^®® along with summary 
information provided by the Research In- 



194. American Educational Research Associa- 
tion, American Psychological Association, and 
National Council on Measurement in Education 
(1999). 

195. Eor example, the National Mathematics Advi- 
sory Panel (2008). 

196. Gersten et al. (2005); Fuchs, Fuchs, Compton 
et al. (2007); Foegen et al. (2007). 



stitute on Progress Monitoring^®^ and the 
National Center on Progress Monitoring 
were also used.^®® 

The studies of screening measures all 
used appropriate correlational designs.^®® 
In many cases, the criterion variable was 
some type of standardized assessment, 
often a nationally normed test (such as 
the Stanford Achievement Test) or a state 
assessment. In a few cases, however, the 
criterion measure was also tightly aligned 
with the screening measure.^®® The latter 
set is considered much weaker evidence 
of validity. 

Studies also addressed inter-tester 
reliability,^®^ internal consistency,^®^ test- 
retest reliability, 2®® and alternate form reli- 
ability.^®^ Many researchers discussed the 
content validity of the measure.^®® A few 
even discussed the consequential valid- 
ity^o®— the consequences of using screen- 
ing data as a tool for determining what 
requires intervention. ^®^ However, these 
studies all used standardized achievement 
measures as the screening measure. 

In recent years, a number of studies of 
screening measures have also begun to 



197. http://www.progressmonitoring.net/. 

198. www.studentprogress.org. 

199. Correlational studies are not eligible for 
WWC review. 

200. For example, Bryant, Bryant, Gersten, Scam- 
macca, and Chavez (2008). 

201. For example, Fuchs et al. (2003a). 

202. For example, Jitendra et al. (2005). 

203. For example, VanDerHeyden, Witt, and Gil- 
bertson (2003). 

204. For example, Thurber, Shinn, and Smol- 
kowski (2002). 

205. For example, Clarke and Shinn (2004); Ger- 
sten and Chard (1999); Foegen, Jiban, and Deno 
(2007). 

206. Messick (1988); Gersten, Keating, and Irvin 
(1995). 

207. For example, Compton, Fuchs, and Fuchs 
(2007). 
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report sensitivity and specificity data.^°® 
Because sensitivity and specificity pro- 
vide information on the false positive and 
false negative rates respectively, they are 
critical in determining the utility of a mea- 
sure used in screening decisions linked 
to resource allocation. Note that work on 
sensitivity and specificity in educational 
screening is in its infancy and no clear 
standards have been developed. 

The remainder of this section presents 
evidence in support of the recommenda- 
tion. We discuss the evidence for measures 
used in both the early elementary and 
upper elementary grades and conclude 
with a more in-depth example of a screen- 
ing study to illustrate critical variables to 
consider when evaluating a measure. 

Summary of evidence 

In the early elementary grades, mea- 
sures examined included general out- 
come measures reflecting a sampling of 
objectives for a grade level that focused 
on whole numbers and number sense. 
These included areas of operations and 
procedures, number combinations or basic 
facts, concepts, and applications. Mea- 
sures to assess different facets of number 
sense — including measures of rote and 
strategic counting, identification of numer- 
als, writing numerals, recognizing quanti- 
ties, and magnitude comparisons — were 
also prevalent. Some research teams 
developed measures focused on a single 
aspect of number sense (such as strategic 
counting),2ii and others developed batter- 
ies to create a composite score from single 



208. Locuniak and Jordan (2008); VanDerHey- 
den et al. (2001); Fuchs, Fuchs, Compton et al. 
(2007). 

209. Fuchs, Fuchs, Compton et al. (2007). 

210. Gersten, Clarke, and Jordan (2007); Fuchs, 
Fuchs, and Compton et al. (2007). 

211. Clarke and Shinn (2004). 



proficiency measures. Still others de- 
veloped a broader measure that assessed 
multiple proficiencies in their screening.^i^ 
An example of a single proficiency embed- 
ded in a broader measure is having stu- 
dents compare magnitudes of numbers. 
As an individual measure, magnitude com- 
parison has predictive validity in the .50 
to .60 range, but having students make 
magnitude comparisons is also included in 
broader measures. For example, the Num- 
ber Knowledge Test (NKT)^^^ requires stu- 
dents to name the greater of two verbally 
presented numbers and includes problems 
assessing strategic counting, simple addi- 
tion and subtraction, and word problems. 
The broader content in the NKT provided 
stronger evidence of predictive validity^!® 
than did single proficiency measures. 

Further information on the characteristics 
and technical adequacy of curriculum- 
based measures (CBM) for use in screen- 
ing in the elementary grades was summa- 
rized by Foegen, Jiban, and Deno (2007). 
They explained that measures primarily 
assessed the objectives of operations or 
the concepts and applications standards 
for a specific grade level. A smaller num- 
ber of measures assessed fluency in basic 
facts, problem solving, or word problems. 
Measures were timed and administration 
time varied between 2 and 6 minutes for 
operations probes and 2 to 8 minutes for 
concepts and applications. Reliability evi- 
dence included test-retest, alternate form, 
internal consistency, and inter-scorer, 
with most reliabilities falling between .80 
and .90, meeting acceptable standards 
for educational decisionmaking. Similar 
evidence was found for validity with most 



212. Bryant, Bryant, Gersten, Scammacca, and 
Chavez (2008). 

213. Okamoto and Case (1996). 

214. Lembke et al. (2008); Clarke and Shinn 
(2004); Bryant, Bryant, Gersten, Scammacca, and 
Chavez (2008). 

215. Okamoto and Case (1996). 

216. Chard et al. (2005). 
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concurrent validity coefficients in the .50 
to .70 range. Lower coefficients were found 
for basic fact measures ranging from .30 
to .60. Researchers have also begun to de- 
velop measures that validly assess magni- 
tude comparison, estimation, and prealge- 
bra proficiencies. 

A study of evaluating a mathematics 
screening instrument — Locuniak and 
Jordan (2008) 

A recent study by Locuniak and Jordan 
(2008) illustrates factors that districts 
should consider when evaluating and se- 
lecting measures for use in screening. The 
researchers examined early mathematics 
screening measures from the middle of 
kindergarten to the end of second grade. 
The two-year period differs from many 
of the other screening studies in the area 
by extending the interval from within a 
school year (fall to spring) to across sev- 
eral school years. This is critical because 
the panel believes the longer the interval 
between when a screening measure and a 
criterion measure are administered, the 
more schools can have confidence that stu- 
dents identified have a significant deficit 
in mathematics that requires intervention. 
The Locuniak and Jordan (2008) study also 
went beyond examining traditional indices 
of validity to examine specificity and sen- 
sitivity. Greater sensitivity and specificity 
of a measure ensures that schools provide 
resources to those students truly at risk 
and not to students misidentified 

The various measures studied by Lo- 
cuniak and Jordan (2008) also reflected 
mathematics content that researchers 
consider critical in the development of 
a child’s mathematical thinking and that 
many researchers have devised screening 



217. Foegen et al. (2007). 



measures to assess. Included were number 
sense measures that assessed knowledge 
of counting, number combinations, non- 
verbal calculation, story problems, num- 
ber knowledge, and short and working 
memory. The authors used block regres- 
sion to examine the added value of the 
math measures in predicting achievement 
above and beyond measures of cognition, 
age, and reading ability (block 1), which 
accounted for 26 percent of the variance 
on 2nd grade calculation fluency. Adding 
the number sense measures (block 2) in- 
creased the variance explained to 42 per- 
cent. Although the research team found 
strong evidence for the measures assess- 
ing working memory (digit span), number 
knowledge, and number combinations, 
the array of measures investigated is in- 
dicative that the field is still attempting to 
understand which critical variables (math- 
ematical concepts) best predict future dif- 
ficulty in mathematics. A similar process 
has occurred in screening for reading dif- 
ficulties where a number of variables (such 
as alphabetic principle) are consistently 
used to screen students for reading dif- 
ficulty. Using the kindergarten measures 
with the strongest correlations to grade 2 
mathematics achievement (number knowl- 
edge and number combinations), the re- 
searchers found rates of .52 for sensitivity 
and .84 for specificity. 

Another feature that schools will need 
to consider when evaluating and select- 
ing measures is whether the measure is 
timed. The measures studied by Locuniak 
and Jordan (2008) did not include a timing 
component. In contrast, general outcome 
measures include a timing component. 
No studies were found by the panel that 
examined a timed and untimed version of 
the same measure. 



218. Deno (1985). 
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Recommendation 2. 

Instructional materials for students 
receiving interventions should 
focus intensely on in-depth 
treatment of whole numbers in 
kindergarten through grade 5 and 
on rational numbers in grades 4 
through 8. These materials should 
be selected by committee. 

Level of evidence: Low 

The panel based this recommendation 
on professional opinion; therefore, the 
evidence rating is low. The professional 
opinion included not only the views of 
the panel members, but also several re- 
cent consensus documents that reflect 
input from mathematics educators and 
research mathematicians involved in is- 
sues related to K-12 mathematics educa- 
tion. Each of these documents was in- 
fluenced to some extent by comparisons 
of curricula standards developed by the 
50 states in the United States with nations 
with high scores on international tests of 
mathematics performance, such as the 
Trends in International Mathematics and 
Science Study (TIMSS) and the Program for 
International Student Assessment (PISA) 
(including the Czech Republic, Flemish 
Belgium, Korea, and Singapore).^^° We note, 
however, that these international compari- 
sons are merely descriptive and thus do 
not allow for causal inferences. In other 
words, we do not know whether their more 
focused curricula or other factors contrib- 
ute to higher performance. 

We note that some of the other reports 
we describe here do not directly address 
the needs of students who receive in- 
terventions to boost their knowledge of 



219. Milgram and Wu (2005); National Council of 
Teachers of Mathematics (2006); National Math- 
ematics Advisory Panel (2008). 

220. For more information on the TIMSS, see 
http://nces.ed.gov/timss/. For more information 
on PISA, see www.oecd.org. 



foundational concepts and procedures. 
However, we concluded that the focus on 
coverage of fewer topics in more depth, 
and with coherence, advocated for general 
education students is as important, and 
probably more important, for students 
who struggle with mathematics. We could 
not locate any experimental research that 
supported our belief, however. Therefore, 
we indicate clearly that we are reflecting 
a growing consensus of professional opin- 
ion, not a convergent body of scientific 
evidence — and conclude that the level of 
evidence is low. 

Summary of evidence 

Three seminal publications were con- 
sulted in forming our opinion. 221 Milgram 
and Wu (2005) were among the first to 
suggest that an intervention curriculum 
for at-risk students should not be over- 
simplified and that in-depth coverage of 
key topics and concepts involving whole 
numbers and then rational numbers was 
critical for future success in mathematics. 
They stressed that mastery of this mate- 
rial was critical, regardless of how long it 
takes. Many before had argued about the 
importance of mastery of units before 
proceeding forward. 222 Milgram and Wu 
argued that stress on precise definitions 
and abstract reasoning was “even more 
critical for at-risk students” (p. 2). They 
acknowledged this would entail extensive 
practice with feedback and considerable 
instructional time. 

The National Council of Teachers of Math- 
ematics Curriculum Tocal Points (2006) 
made a powerful statement about reform 
of mathematics curriculum for all students 
by calling for the end of brief ventures into 
many topics in the course of a school year. 



221. Milgram and Wu (2005); National Council of 
Teachers of Mathematics (2006); National Math- 
ematics Advisory Panel (2008). 

222. For example, Bloom (1980); Guskey (1984); 
Sllbert, Gamine, and Stein (1989). 
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The topics it suggests emphasize whole 
numbers (properties, operations, prob- 
lem solving) and especially fractions and 
related topics involving rational numbers 
(proportion, ratio, decimals). The report is 
equally clear that algorithmic proficiency 
is critical for understanding properties 
of operations and related concepts and 
that algorithmic proficiency, quick re- 
trieval of mathematics facts, and in-depth 
knowledge of such concepts as place value 
and properties of whole numbers are all 
equally important instructional goals. This 
position was reinforced by the report of 
the National Mathematics Advisory Panel 
(2008) two years later, which provided de- 
tailed benchmarks and again emphasized 
in-depth coverage of key topics involving 
whole numbers and rational numbers as 
crucial for all students. 

In the view of the panel, students in inter- 
vention programs need to master material 
on whole numbers and rational numbers, 
and they must ultimately work with these 
concepts and principles at an abstract 
level. We feel that it is less important for 
4th graders in an intervention program 
to cover the entire scope and sequence 
of topics from the year before. Instead, 
the aim is to cover the key benchmarks 
articulated in the National Mathematics 
Advisory Panel report, involving whole 
numbers and rational numbers that stu- 
dents do not fully grasp, and build profi- 
ciencies they lack. 

Recommendatiori 3. Instruction 
during the intervention should 
be explicit and systematic. This 
includes providing models of 
proficient problem solving, 
verbalization of thought processes, 
guided practice, corrective feedback, 
and frequent cumulative review. 

Level of evidence: Strong 

The panel judged the level of evidence sup- 
porting the recommendation to be strong. 



The panel found six studies^^^ conducted 
with low achieving or learning disabled 
students^24 between 2nd and 8th grades 
that met WWC standards or met standards 
with reservations and included compo- 
nents of explicit and systematic instruc- 
tion. Appendix table D1 (p. 69) provides 
an overview of the components of explicit 
instruction in each intervention, including 
the use of teacher demonstration (such 
as verbalization during demonstration 
and the use of multiple examples), stu- 
dent verbalization (either as a procedural 
requirement or as a response to teacher 
questions), scaffolded practice, cumula- 
tive review, and corrective feedback. The 
relevant treatment and comparison groups 
compared in each study and the outcomes 
found for each domain are included in the 
table, as are grade-level, typical session 
length, and duration of the intervention. 

Because of the number of high-quality 
randomized and quasi-experimental de- 
sign studies using explicit and systematic 
mathematics instruction across grade lev- 
els and diverse student populations, the 
frequency of significant positive effects, 
and the fact that numerous research teams 
independently produced similar findings, 
the panel concluded that there is strong 
evidence to support the recommendation 
to provide explicit and systematic instruc- 
tion in tier 2 mathematics interventions. 



223. Darch, Carnine, and Gersten (1984); Fuchs 
et al. (2003a); Jitendra et al. (1998); Tournaki 
(2003); Schunk and Cox (1986); Wilson and Sin- 
delar (1991). 

224. These students specifically had difficulties 
with mathematics. 

225. For this practice guide, the components of 
explicit and systematic mathematics instruction 
are identified as providing models of proficient 
problem solving, verbalizing teacher and student 
thought processes, scaffolded practice, cumula- 
tive review, and corrective feedback. 
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Summary of evidence 

Teacher demonstrations and think-aloads. 
The panel suggests that teachers verbal- 
ize the solution to problems as they model 
problem solving for students. Tournaki 
(2003) assessed this approach by compar- 
ing a group of students whose teachers 
had demonstrated and verbalized an ad- 
dition strategy (the Strategy group) against 
a group of students whose teacher did not 
verbalize a strategy (the Drill and Practice 
group). As depicted in appendix table Dl, 
the effects on an assessment of single-digit 
addition were significant, positive, and 
substantial in favor of the students whose 
teacher had verbalized a strategy.^^® 

All six studies examined interventions that 
included teacher demonstrations early in 
the mathematics lessons.^^’ For example, 
Schunk and Cox (1986), Jitendra et al. 
(1998), and Darch, Carnine, and Gersten 
(1984) all conducted studies in which in- 
struction began with the teacher verbaliz- 
ing the steps to solve sample mathemat- 
ics problems. Because this demonstration 
procedure was used to instruct students 
in both treatment and comparison groups, 
the effects of this component of explicit in- 
struction cannot be evaluated from these 
studies. However, the widespread use of 
teacher demonstration in interventions 
that include other components of explicit 
instruction supports the panel’s conten- 
tion that this is a critical component of 
explicit instructional practice. 

For teacher demonstration, the panel spe- 
cifically recommends that teachers provide 



226. Note that during the intervention, students 
in the Strategy condition were also encouraged to 
verbalize the problem-solving steps and that this 
may also be a factor in the success of the inter- 
vention. The Tournaki (2003) study is described 
in more detail below. 

227. Darch, Carnine, and Gersten (1984); Fuchs 
et al. (2003a); Jitendra et al. (1998); Tournaki 
(2003); Schunk and Cox (1986); Wilson and Sin- 
delar (1991). 



numerous models of solving easy and hard 
problems proficiently. Demonstration with 
easy and hard problems and the use of 
numerous examples were not assessed 
as independent variables in the studies 
reviewed. However, Wilson and Sindelar 
(1991) did use numerous examples in in- 
struction for both groups evaluated. The 
key difference between the groups was that 
students in the treatment group were ex- 
plicitly taught problem-solving strategies 
through verbal and visual demonstrations 
while students in the comparison group 
were not taught these strategies. This study 
demonstrated substantively important pos- 
itive effects with marginal significance in 
favor of the treatment group. 

Scaffolded practice. Scaffolded practice, 
a transfer of control of problem solving 
from the teacher to the student, was a 
component of mathematics interventions 
in four of the six studies. ^29 In each study, 
the intervention groups that included scaf- 
folded practice demonstrated significant 
positive effects; however, it is not possible 
to parse the effects of scaffolded instruc- 
tion from the other components of ex- 
plicit instruction in these multicomponent 
interventions. 

Student verbalization. Three of the six 
studies23° included student verbalization 
of problem-solution steps in the interven- 
tions. For example, Schunk and Cox (1986) 
assessed the effect of having students ver- 
balize their subtraction problem-solving 
steps versus solving problems silently. 
There were significant and substantial 
positive effects in favor of the group that 



228. For this guide, the panel defined margin- 
ally significant as a p-value in the range of .05 
to .10. Following WWC guidelines, an effect size 
greater than 0.25 is considered substantively 
important. 

229. Darch, Carnine, and Gersten (1984); Fuchs 
et al. (2003a); Jitendra et al. (1998); Tournaki 
(2003). 

230. Schunk and Cox (1986); Jitendra et al. (1998); 
Tournaki (2003). 
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verbalized steps. The Tournaki (2003) 
intervention also included student ver- 
balization among other components and 
had significant positive effects. Among 
other intervention components, Jitendra 
et al. (1998) included student verbalization 
through student responses to a teacher’s 
facilitative questions. Again, the effects 
were substantively important or statisti- 
cally significant and positive, but they can- 
not be attributed to a single component in 
this multi component intervention. 

Corrective feedback. Four of the six stud- 
ies included immediate corrective feed- 
back in the mathematics interventions. 
For example, in the Darch, Carnine, and 
Gersten (1984) study, when a student 
made an error, teachers in the treatment 
group would first model the appropriate 
response, then prompt the students with 
questions to correct the response, then 
reinforce the problem-solving strategy 
steps again. In three of the studies, the 
effects of the corrective feedback compo- 
nent cannot be isolated from the effects 
of the other instructional components; 
however, the effects of the interventions 
including corrective feedback were posi- 
tive and significant. 

Cumulative review. The panel’s assertion 
that cumulative review is an important 
component of explicit instruction is based 
primarily on expert opinion because only 
one study in the evidence base included 
cumulative review as a component of the 
intervention. This study had positive sig- 
nificant effects in favor of the instructional 
group that received explicit instruction in 
strategies for solving word problems. 



231. Darch, Carnine, and Gersten (1984); Jiten- 
dra et al. (1998); Tournaki (2003); Schunk and 
Cox (1986). 

232. Darch, Carnine, and Gersten (1984); Jitendra 
et al. (1998); Tournaki (2003). 

233. Fuchs et al. (2003a). 



In summary, the components of explicit 
and systematic instruction are consistently 
associated with significant positive effects 
on mathematics competency, most often 
when these components are delivered in 
combination. An example of a study that 
examines the effects of a combination of 
these components is described here. 

A study of explicit and systematic 
instruction— Tournaki (2003) 

Explicit and systematic instruction is a 
multicomponent approach, and an in- 
tervention examined in Tournaki (2003) 
exemplifies several components in com- 
bination. This study was conducted with 
42 students in grade 2 special education 
classrooms. 234 The students, between 8 
and 10 years old, were classified as learn- 
ing disabled with weaknesses in both read- 
ing and mathematics. Twenty-nine were 
boys, and 13 were girls. 

Prior to the intervention, the students 
completed a pretest assessment consist- 
ing of 20 single-digit addition problems 

(such as 6 -E 3 = ). Internal consistency 

of the assessment was high (Cronbach’s 
alpha of .91). Student performance on the 
assessment was scored for accuracy and 
latency (the time it took each student to 
complete the entire assessment). The ac- 
curacy score is a measure of student abil- 
ity to perform mathematical operations, 
and the latency score is an indication of 
student fluency with single-digit addition 
facts. After the intervention, students 
completed a posttest assessment that was 
identical to the pretest. 

Students were randomly assigned to one of 
three groups (two instruction groups and 



234. The sample also included 42 grade 2 stu- 
dents from general education classrooms, but 
only the results for the special education students 
are presented as relevant to this practice guide. 
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a comparison group). Students in the 
two instruction groups met individually 
with a graduate assistant for a maximum 
of eight 15-minute supplemental math- 
ematics sessions on consecutive school 
days. Instructional materials for both 
groups consisted of individual worksheets 
for each lesson with a group of 20 sin- 
gle-digit addition problems covering the 
range from 2 -e 2 = to 9 -e 9 = . 

The Strategy instruction group received 
explicit and systematic instruction to im- 
prove fact fluency. The instruction began 
with the teacher modeling the minimum 
addend strategy for the students and 
thinking aloud. This strategy is an ef- 
ficient approach for solving single-digit 

addition problems (such as 5 -e 3 = ). 

The teacher began by saying, “When 1 get 
a problem, what do 1 do? 1 read the prob- 
lem; 5 plus 3 equals how many? Then 1 find 
the smaller number.” Pointing to the num- 
ber, the teacher says, “Three. Now 1 count 
fingers. How many fingers am 1 going to 
count? Three.” The teacher counts three 
fingers, points to the larger number and 
says, “Now, starting from the larger num- 
ber, 1 will count the fingers.” The teacher 
points to the 5, then touches each finger 



235. Students in the comparison group received 
only the pretest and posttest without any supple- 
mental mathematics instruction outside their 
classroom. Because the scope of the practice 
guide is examining the effects of methods of 
teaching mathematics for low-achieving stu- 
dents, the comparison group findings are not 
included here. 



as she says, “5, 6, 7, 8. How many did 1 
end up with? Eight. I’ll write 8 to the right 
of the equal sign.” After writing the num- 
ber, the teacher finishes modeling by say- 
ing, “I’ll read the whole problem; 5 plus 3 
equals 8.” 

The teacher and student solved two prob- 
lems together through demonstration and 
structured probing. The student was then 
asked to solve a third problem indepen- 
dently while verbalizing the strategy steps 
aloud. When a student made an error, the 
teacher gave corrective feedback. The stu- 
dent was asked to solve the remaining prob- 
lems without verbalization and to work 
as fast as possible, but when an error oc- 
curred, the teacher interrupted the lesson 
and reviewed the steps in the strategy. 

Students in the Drill and Practice group 
were asked to solve the problems as 
quickly as possible. At the completion of 
each lesson, the teacher marked the stu- 
dent’s errors and asked the student to re- 
compute. If the error persisted, the teacher 
told the student the correct answer. Re- 
sults indicate significant and substantial 
positive effects in favor of the Strategy 
group, which received explicit and sys- 
tematic instruction, relative to Drill and 
Practice group, which received a more tra- 
ditional approach. In this study, the com- 
bination of teacher demonstration, student 
verbalization, and corrective feedback was 
successful in teaching students with math- 
ematics difficulties to accurately complete 
single-digit addition problems. 
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effect size is not significant (n.s.). 

Source: Authors’ analysis based on studies in table. 
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Recommendation 4. Interventions 
should include instruction on 
solving word problems that is based 
on common underlying structures. 

Level of evidence: Strong 

The panel rated the level of evidence as 
strong. We located nine studies that met the 
standards of the WWC or met the standards 
with reservations and demonstrated sup- 
port for the practice of teaching students to 
solve word problems based on their under- 
lying structures.^3® Appendix table D2 (p. 73) 
provides an overview of each of the inter- 
ventions examined in these nine studies. 

In all nine interventions, students were 
taught to recognize the structure of a word 
problem in order to solve it, and they were 
taught how to solve each problem type.^^^ 
Six of the studies took the instruction on 
problem structure a step further. Students 
were taught to distinguish superficial from 
substantive information in word problems 
in order to transfer solution methods from 
familiar problems they already knew how 
to solve to problems that appeared unfamil- 
iar.2^® Because of the large number of high- 
quality randomized studies conducted that 
examined this practice and because most 
of the interventions examined led to sig- 
nificant and positive effects on word prob- 
lem outcomes for children designated as 
low achieving and/or learning disabled. 



236. Jitendra et al. (1998); Xin, Jitendra, and Deat- 
line-Buchman (2005); Darch, Carnine, and Gersten 
(1984); Fuchs et al. (2003a); Fuchs et al. (2003b); 
Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, 
Finelli et al. (2004); Fuchs, Seethaler et al. (2008); 
Fuchs, Fuchs, Craddock et al. (2008). 

237. Jitendra et al. (1998); Xin, Jitendra, and Deat- 
line-Buchman (2005); Darch, Carnine, and Gersten 
(1984); Fuchs et al. (2003a); Fuchs et al. (2003b); 
Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, 
Finelli et al. (2004); Fuchs, Seethaler et al. (2008); 
Fuchs, Fuchs, Craddock et al. (2008). 

238. Fuchs et al. (2003a); Fuchs et al. (2003b); 
Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, 
Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. 
(2008); Fuchs, Seethaler et al. (2008). 



we conclude that there is strong evidence 
to support this recommendation. 

Summary of evidence 

Teach the structure of problem types. In 
three of the studies, students were taught 
to identify problems of a given type and 
then to design and execute appropriate 
solution strategies for each type.^^s in one 
of these interventions, students learned to 
represent the problem using a schematic 
diagram. 240 Once students learned to iden- 
tify the key problem features and map 
the information onto the diagram, they 
learned to solve for unknown quantities in 
word problems while still representing the 
problem using a schematic diagram. This 
intervention had significant and positive 
effects on a word problem outcome based 
on a test of problems similar to those 
taught during the intervention. 

In another intervention that also led to a 
significant and positive effect on a word 
problem outcome, students were taught to 
discriminate between multiplication and 
addition problems, and between multiplica- 
tion and division problems. To discrimi- 
nate multiplication from addition problems, 
students were taught that if a problem asks 
them to use the same number multiple times 
(sometimes signaled by the words “each” 
and "every”) to obtain the total number, 
the problem requires multiplication. If the 
problem does not ask the student to use the 
same number multiple times to obtain the 
total number, the problem requires addition. 
Next, after students learned the relationship 
between multiplication and division through 
the concept of number families, they learned 
to multiply when the two smaller numbers 
are given without the big number and to 
divide when the big number is given. 



239. Jitendra et al. (1998); Xin, Jitendra, and 
Deatline-Buchman (2005); Darch, Carnine, and 
Gersten (1984). 

240. Xin, Jitendra, and Deatline-Buchman (2005). 

241. Darch, Carnine, and Gersten (1984). 
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Transfer solution methods from familiar 
problem types to problems that appear un- 
familiar. In addition to teaching students 
to recognize and solve different problem 
types, six of these studies taught students 
how to transfer solution methods to prob- 
lems that appear different but really re- 
quire the same solution methods as those 
they already know how to solve. In each 
of these interventions, students were first 
taught the pertinent structural features 
and problem-solution methods for differ- 
ent problem types. Next, they were taught 
about superficial problem features that can 
change a problem without altering its struc- 
ture or solution (for example, different for- 
mat, different key vocabulary, additional or 
different question, irrelevant information) 
and how to solve problems with varied 
cover stories and superficial features. 

In all six studies, word problem outcome 
measures ranged from those where the 
only source of novelty was the cover story 
(immediate transfer), to those that varied 
one or more superficial features (near or 
far transfer). In five cases^^^ the average 
impact of the intervention on these out- 
come measures was positive and signifi- 
cant for the samples designated as low 
achieving and/or learning disabled, and 
in one case,^^^ the impact was marginally 
significant. These studies show that in- 
struction on problem structure and trans- 
ferring known solution methods to unfa- 
miliar problems is consistently associated 
with marginally or statistically significant 
positive effects on word problem solving 
proficiencies for students experiencing 
mathematics difficulties. 



242. Fuchs et al. (2003a); Fuchs et al. (2003b); 
Fuchs, Fuchs, Prentice et al. (2004); Fuchs, Fuchs, 
Finelli et al. (2004); Fuchs, Fuchs, Craddock et al. 
(2008); Fuchs, Seethaler et al. (2008). 

243. Fuchs et al. (2003a); Fuchs et al. (2003b); Fuchs, 
Fuchs, Prentice et al. (2004); Fuchs, Fuchs, Finelli et 
al. (2004); Fuchs, Fuchs, Craddock et al. (2008). 

244. Fuchs, Seethaler et al. (2008). 



A study of teaching students 
to transfer solution methods— Fuchs, 
Fuchs, Finelli, Courey, and Hamlett 
( 2004 ) 

Fuchs and colleagues (2004) conducted 
a study that investigated the effects of 
teaching students how to transfer known 
solution methods to problems that are 
only superficially different from those 
they already know how to solve. The 
authors randomly assigned 24 teachers to 
three groups: 1) transfer instruction, 2) ex- 
panded transfer instruction, and 3) regular 
basal instruction (comparison group). 
The 351 students in these 24 classes that 
were present for each of the pretests and 
posttests were participants in the study. 

The intervention included 25- to 40-minute 
lessons, approximately twice per week for 
17 weeks.2^^ Students in the expanded trans- 
fer condition learned basic math problem- 
solving strategies in the first unit of instruc- 
tion (six sessions over three weeks). They 
were taught to verify that their answers 
make sense; line up numbers from text to 
perform math operations; check operations; 
and label their work with words, monetary 
signs, and mathematical symbols. 

The remaining units each focused on one of 
four problem types: 1) shopping list prob- 
lems (buying multiple quantities of items, 
each at a different price); 2) buying bag prob- 
lems (determining how many bags contain- 
ing a specified number of objects are needed 
to come up with a desired total number of 



245. Fuchs, Fuchs, Finelli et al. (2004). 

246. Since the comparison between the expanded 
transfer condition and the control condition (reg- 
ular basal instruction) is most relevant to this 
practice guide, we do not discuss the transfer 
instruction condition here. 

247. Although this intervention was taught in a 
whole-class format, the authors reported sepa- 
rate effects for students classified as low achiev- 
ing and for students classified as learning dis- 
abled; therefore, the results are relevant to this 
practice guide. 
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objects); 3) half problems (determining what 
half of some total amount is); and 4) picto- 
graph problems (summing two addends, 
one derived from a pictograph). There were 
seven sessions within each unit. 

In sessions one through four, with the 
help of a poster listing the steps, students 
learned problem-solution rules for solving 
the type of problem being taught in that 
particular unit. In the first session, teachers 
discussed the underlying concepts related 
to the problem type, presented a worked 
example, and explained how each step of 
the solution method was applied in the 
example. After presenting several worked 
examples, the teachers presented partially 
worked examples while the students ap- 
plied the steps of the solution method. Stu- 
dents then completed one to four problems 
in pairs. Sessions two through four were 
similar, but more time was spent on par- 
tially worked examples and practice, and 
at the end of each session, students com- 
pleted a problem independently. 

In sessions five and six, teachers taught 
students how to transfer the solution meth- 
ods using problems that varied cover sto- 
ries, quantities, and one transfer feature 
per problem. In session five, the teachers 
began by explaining that transfer means 
to move and presented examples of how 
students transfer skills. Then, teachers 
taught three transfer features that change 
a problem without changing its type or 
solution, including formatting, unfamiliar 
vocabulary, and posing a different ques- 
tion. These lessons were facilitated by a 
poster displayed in the classroom about 
the three ways problems change. Again, 
teachers presented the information and 
worked examples, and moved gradually 
to partially worked examples and prac- 
tice in pairs. Session six was similar to 
session five, but the students spent more 
time working in pairs, and they completed 
a transfer problem independently. 



In the seventh session, teachers instructed 
students on three additional superficial 
problem features including irrelevant infor- 
mation, combining problem types, and mix- 
ing superficial problem features. Teachers 
taught this lesson by discussing how prob- 
lems encountered in “real life” incorporate 
more information than most problems that 
the students know how to solve. They used 
a poster called Real-Life Situations to illus- 
trate each of these superficial problem fea- 
tures with a worked example. Next, students 
worked in pairs to solve problems that var- 
ied real-life superficial problem features and 
then completed a problem independently. 

The authors used four measures to deter- 
mine the results of their intervention on 
word problem-solving proficiencies. The 
first measure used novel problems struc- 
tured the same way as problems used in 
the intervention. The second incorporated 
novel problems that varied from those used 
in instruction in terms of the look or the 
vocabulary or question asked. The third 
incorporated novel problems that varied 
by the three additional transfer features 
taught in session seven. The fourth was a 
measure designed to approximate real-life 
problem solving. Although this intervention 
was taught in a whole-class format, the au- 
thors separated results for students classi- 
fied as low performing^^® and for students 
classified as learning disabled. The average 
impacts on these four outcome measures 
were positive and significant for both the 
sample designated as low performing and 
the sample designated as learning disabled. 
It is notable that the intervention had a posi- 
tive and significant impact on the far trans- 
fer measure (the measure that approximated 
real-life problem solving). This study dem- 
onstrates a successful approach for instruct- 
ing students with mathematics difficulties 
on solving word problems and transferring 
solution methods to novel problems. 



248. Using pretest scores on the first transfer 
problem-solving measure, the authors desig- 
nated each student as low performing, average 
performing, or high performing. 
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Table D2. Studies of interventions that taught students to discriminate problem types that met WWC standards 
(with or without reservations) 
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APPENDIX D. TECHNICAL INFORMATION ON THE STUDIES 



Recommendation 5. Intervention 
materials should include 
opportunities for students to 
work with visual representations 
of mathematical ideas and 
interventionists should be proficient 
in the use of visual representations 
of mathematical ideas. 

Level of evidence: Moderate 

The panel judged the level of evidence 
for this recommendation to be moderate. 
We found 13 studies conducted with stu- 
dents classified as learning disabled or low 
achieving that met WWC standards or met 
standards with reservations. Four in 
particular examined the impact of tier 2 in- 
terventions against regular tier 1 instruc- 
tion. Appendix table D3 (p. 77) provides 
an overview of these 13 studies. Note that 
in an attempt to acknowledge meaning- 
ful effects regardless of sample size, the 
panel followed WWC guidelines and con- 
sidered a positive statistically significant 
effect, or an effect size greater than 0.25, 
as an indicator of positive effects. 

Summary of evidence 

The representations in 11 of the 13 studies 
were used mainly to teach word problems 
and concepts (fractions and prealgebra).^^^ 



249. Artus and Dyrek (1989); Darch, Carnine, and 
Gersten (1984); Fuchs, Powell et al. (2008); Fuchs, 
Fuchs, Craddock et al. (2008); Fuchs, Seethaler et 
al. (2008); Fuchs et al. (2005); Jitendra et al. (1998); 
Butler et al. (2003); Walker and Poteet (1989); Wil- 
son and Sindelar (1991); Witzel, Mercer, and Miller 
(2003); Witzel (2005); Woodward (2006). 

250. Fuchs, Powell et al. (2008); Fuchs, Fuchs, 
Craddock et al. (2008); Fuchs, Seethaler et al. 
(2008); Fuchs et al. (2005). 

251. For more details on WWC guidelines for 
substantively important effects, see the What 
Works Clearinghouse Procedures and Standards 
Handbook (WWC, 2008). 

252. Jitendra et al. (1998); Butler et al. (2003); Wit- 
zel (2005); Darch, Carnine, and Gersten (1984); 
Fuchs, Powell et al. (2008); Fuchs, Fuchs, Crad- 
dock et al. (2008); Fuchs, Seethaler et al. (2008); 



In one study, visual representations were 
used to teach mathematics facts. in all 
13 studies, representations were used to 
understand the information presented in 
the problem. Specifically, the representa- 
tions helped answer such questions as 
what type of problem it is and what oper- 
ation is required. In all 13 studies, visual 
representations were part of a complex 
multicomponent instructional interven- 
tion. Therefore, it is not possible to ascer- 
tain the role and impact of the representa- 
tion component. 

Of the 13 studies, 4 used visual represen- 
tations, such as drawings or other forms 
of pictorial representations, to scaffold 
learning and pave the way for the under- 
standing of the abstract version of the 
representation. 254 jitendra et al. (1998) 
examined the differential effects of two 
instructional strategies, an explicit strat- 
egy using visual representations and a 
traditional basal strategy. Students were 
taught explicitly to identify and differenti- 
ate among word problems types and map 
the features of the problem onto the given 
diagrams specific to each problem type. 
The intervention demonstrated a nonsig- 
nificant substantively important positive 
effect. Wilson and Sindelar (1991) used a 
diagram to teach students the “big num- 
ber” rule (e.g., when a big number is given, 
subtract) (ES = .82~). Woodward (2006) ex- 
plored the use of visuals such as a number 
line to help students understand what an 

abstract fact such as 6 x 7 = meant. 

The study yielded a substantively impor- 
tant positive effect on mathematics facts, 
and a positive and marginally significant 
average effect on operations. 



Fuchs et al. (2005); Walker and Poteet (1989); 
Wilson and Sindelar (1991); Witzel, Mercer, and 
Miller (2003). 

253. Woodward (2006). 

254. Jitendra et al. (1998); Walker and Poteet (1989); 
Wilson and Sindelar (1991); Woodward (2006). 
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Three studies used manipulatives in the 
early stages of instruction to reinforce un- 
derstanding of basic concepts and opera- 
tions. For example, Darch et al. (1984) 
used concrete models such as groups of 
boxes to teach rules for multiplication 
problems. Similarly, Fuchs, Fuchs, Crad- 
dock. et al. (2008) used manipulatives in 
their tutoring sessions to target and teach 
the most difficult concepts observed in 
the classroom. In another study, Fuchs, 
Seethaler et al. (2008) used concrete ma- 
terials and role playing to help students 
understand the underlying mathemati- 
cal structure of each problem type. In all 
these studies, manipulatives were one as- 
pect of a complex instructional package. 
The studies resulted in mostly significant 
positive domain average effect sizes in the 
range of .60 to 1.79. 

In six studies, both concrete and visual 
representations were used to promote 
mathematical understanding.^^® For exam- 
ple, Artus and Dyrek (1989) used concrete 
objects (toy cars, empty food wrappers) 
and visuals (drawings) to help students 
understand the story content, action, and 
operation in the word problems (ES = .87~). 
Likewise, Fuchs, Powell et al. (2008) used 
manipulatives in the initial stages and later 
pictorial representations of ones and tens 
in their software program (ES = .55, n.s.). 
However, in both studies, concrete objects 
and visual representations were not part 
of an instructional format that promoted 
systematic scaffolded learning. In other 
words, instruction did not include fading 
the manipulatives and visual representa- 
tions to promote understanding of math 
at the more typical abstract level. 

In the remaining four studies, manipu- 
latives and visual representations were 



255. Darch et al. (1984); Fuchs, Seethaler et al. 
(2008); Fuchs, Fuchs, Craddock et al. (2008). 

256. Artus and Dyrek (1989); Butler et al. (2003); 
Fuchs, Powell et al. (2008); Fuchs et al. (2005); Wit- 
zel (2005); Witzel, Mercer, and Miller (2003). 



presented to the students sequentially to 
promote scaffolded instruction. ^^7 This 
model of instruction, with its underpin- 
ning in Bruner’s (1966) work, is referred 
to as a concrete to representation to ab- 
stract (CRA) method of instruction. The 
CRA method is a process by which stu- 
dents learn through the manipulation of 
concrete objects, then through visual rep- 
resentations of the concrete objects, and 
then by solving problems using abstract 
notation. 258 Puchs et al. (2005) taught 1st 
grade students basic math concepts (e.g., 
place value) and operations initially using 
concrete objects, followed by pictorial rep- 
resentations of blocks, and finally at the 

abstract level (e.g., 2 + 3 = ) without the 

use of manipulatives or representations. 
Butler et al. (2003) examined the differen- 
tial impact of using two types of scaffolded 
instruction for teaching fractions, one 
that initiated scaffolding at the concrete 
level (concrete-representation-abstract) 
and the other that started at the repre- 
sentation level (representation-abstract). 
Neither variation resulted in significant 
differences. 

Witzel (2005) and Witzel, Mercer, and Miller 
(2003) investigated the effectiveness of 
the scaffolded instruction using the CRA 
method to teach prealgebra (e.g., X - 4 = 
6) to low-achieving students and students 
with disabilities. Using an explicit instruc- 
tional format, Witzel taught students ini- 
tially using manipulatives such as cups and 
sticks. These were replaced with drawings of 
the same objects and finally faded to typical 
abstract problems using Arabic symbols (as 
seen in most textbooks and standardized 
exams). Both studies resulted in statisti- 
cally significant or substantively important 
positive gains (Witzel, Mercer, and Miller, 
2003 and ES = .83*; Witzel, 2005 and ES = 
.54, n.s.). One of these studies is described 
in more detail here. 



257. Butler et al. (2003); Fuchs et al. (2005); Witzel 
(2005); Witzel, Mercer, and Miller (2003). 

258. Witzel (2005). 
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A study of CRA instruction — Witzel, 
Mercer, and Miller (2003) 

In 2003, Witzel, Mercer, and Miller published 
a study that investigated the effects of using 
the CRA method to teach prealgebra. The 
participants in the study were teachers and 
students in 12 grade 6 and 7 classrooms in 
a southeastern urban county. Each teacher 
taught one of two math classes using CRA 
instruction (treatment group) and the other 
using abstract-only traditional methods 
(traditional instruction group). Of those 
participating, 34 students with disabili- 
ties^®° or at risk for algebra difficulty^®^ in 
the treatment group were matched with 34 
students with similar characteristics across 
the same teacher’s classes in the traditional 
instruction group. 



259. Witzel, Mercer, and Miller (2003). 

260. These students were identified through school 
services as those who needed additional support, 
had a 1.5 standard deviation discrepancy between 
ability and achievement, and had math goals listed 
in their individualized education plans. 

261. These students met three criteria: per- 
formed below average in the classroom according 
to the teacher, scored below the 50th percentile 
in mathematics on the most recent statewide 
achievement test, and had socioeconomically 
disadvantaged backgrounds. 



The students in both groups were taught 
to transform equations with single vari- 
ables using a five-step 19-lesson sequence 
of algebra equations. In each session, 
the teacher introduced the lesson, mod- 
eled the new procedure, guided students 
through procedures, and began to have 
students working independently. For the 
treatment group, these four steps were 
used for instruction at the concrete, rep- 
resentational, and abstract stages of each 
concept. Teachers taught the concrete les- 
sons using manipulative objects such as 
cups and sticks, the representational les- 
sons using drawings of the same objects, 
and the abstract lessons using Arabic sym- 
bols. For the traditional instruction group, 
the teachers covered the same content 
for the same length of time (50 minutes), 
but the teachers used repeated abstract 
lessons rather than concrete objects and 
pictorial representations. 

A 27-item test to measure knowledge on 
single-variable equations and solving for 
a single variable in multiple-variable equa- 
tions was administered to the students one 
week before treatment (pretest), after the 
last day of the treatment (posttest), and 
three weeks after treatment ended (follow- 
up). The CRA intervention had a positive 
and significant effect on knowledge of the 
prealgebra concepts assessed. 
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Recommendation 6. 

Interventions at all grade levels 
should devote about 10 minutes 
in each session to building fluent 
retrieval of basic arithmetic facts. 

Level of evidence: Moderate 

The panel judged the level of evidence 
supporting the recommendation to be 
moderate. We found seven studies con- 
ducted with low-achieving or learning 
disabled students between grades 1 and 
4 that met WWC standards or met stan- 
dards with reservations and included fact 
fluency instruction in an intervention. 
Appendix table D4 (p. 83) provides an over- 
view of the studies and indicates whether 
fact fluency was the core content of the 
intervention or a component of a larger 
intervention. The relevant treatment and 
comparison groups in each study and the 
outcomes for each domain are included 
in the table. Grade level, typical session 
length, and duration of the intervention 
are also in the table. 

Given the number of high-quality ran- 
domized and quasi-experimental design 
studies conducted across grade levels 
and diverse student populations that in- 
clude instruction in fact fluency as either 
an intervention or a component of an in- 
tervention, the frequency of small but 
substantively important or significant 
positive effects on measures of fact flu- 
ency and mathematical operations (effect 
sizes ranged from .11 to 2.21), and the 
fact that numerous research teams inde- 
pendently produced similar findings, the 
panel concluded that there is moderate 
evidence to support the recommendation 
to provide instruction in fact fluency for 
both tier 2 and tier 3 mathematics inter- 
ventions across grade levels. The panel 



262. Fuchs, Fuchs, Hamlett et al. (2006); Fuchs, 
Seethaler et al. (2008); Fuchs et al. (2005). Bernie- 
Smith (1991); Fuchs, Powell et al. (2008); Tournaki 
(2003); Woodward (2006). 



acknowledges that the broader implication 
that general mathematics proficiency will 
improve when fact fluency improves is the 
opinion of the panel. 

Summary of evidence 

The panel recognizes the importance of 
knowledge of basic facts (addition, sub- 
traction, multiplication, and division) for 
students in kindergarten through grade 4 
and beyond. Two studies examined the ef- 
fects of teaching mathematics facts relative 
to the effects of teaching spelling or word 
identification using similar methods. In 
both studies, the mathematics facts group 
demonstrated substantively important or 
statistically significant positive gains in 
facts fluency relative to the comparison 
group, although the effects were significant 
in only one of these two studies. 

Another two interventions included a facts 
fluency component in combination with a 
larger tier 2 intervention. For example, 
in the Fuchs et al. (2005) study, the final 10 
minutes of a 40-minute intervention ses- 
sion were dedicated to practice with addi- 
tion and subtraction facts. In both studies, 
tier 2 interventions were compared against 
typical tier 1 classroom instruction. In each 
study, the effects on mathematics facts 
were not significant. Significant positive ef- 
fects were detected in both studies in the 
domain of operations, and the fact fluency 
component may have been a factor in im- 
proving students’ operational abilities. 

Relationships among facts. The panel sug- 
gests emphasizing relationships among 
basic facts, and five of the studies examined 



263. Fuchs, Fuchs, Hamlett et al. (2006); Fuchs, 
Powell et al. (2008). 

264. In Fuchs, Fuchs, Hamlett et al. (2006), the 
effects on addition fluency were positive while 
there was no effect on subtraction fluency. 

265. Fuchs, Seethaler et al. (2008); Fuchs et al. 
(2005). 
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exemplify this practice.^®® In Woodward 
(2006), the Integrated Strategies group was 
specifically taught the connection between 
single-digit facts (e.g., 4x2 = ) and ex- 
tended facts (40 X 2 = ). In Fuchs et al. 

(2005, 2006c, 2008e), mathematics facts 
are presented in number families (e.g., 
1 + 2 = 3 and 3-2 = 1). Bernie-Smith (1991) 
examined the effects of a counting up/on 
procedure that highlighted the relationship 
between facts versus a rote memorization 
method that did not highlight this relation- 
ship. There was a substantively important 
nonsignificant positive effect in favor of 
the group that was taught the relationship 
between facts. Note that fact relationships 
were not isolated as independent variables 
in this study. 

Materials to teach math facts. The stud- 
ies used a variety of materials to teach 
mathematics facts. Woodward (2006) used 
worksheets, number lines, and arrays of 
blocks projected on overheads to help stu- 
dents visualize fact strategies. Tournaki 
(2003) also used worksheets. Three studies 
included flash cards. For example, the 
Fuchs, Seethaler et al. (2008) intervention 
included flash cards with individual ad- 
dition and subtraction problems on each 
card. Students had up to two minutes to re- 
spond to as many cards as they could, and 
they were provided with corrective feed- 
back on up to five errors each session. 

Three studies included computer assisted 
instruction to teach mathematics facts. 

In all three interventions, students used a 
computer program designed to teach addi- 
tion and subtraction facts. In this program, 
a mathematics fact was presented briefly 
on the computer screen. When the fact dis- 



266. Bernie-Smith (1991); Fuchs et al. (2005); 
Fuchs, Fuchs, Hamlett et al. (2006), Fuchs, Seetha- 
ler et al. (2008); Woodward (2006). 

267. Bernie-Smith (1991); Fuchs, Seethaler et al. 
(2008); Fuchs, Powell et al. (2008). 

268. Fuchs et al. (2005); Fuchs, Fuchs, Hamlett et 
al. (2006); Fuchs, Powell et al. (2008). 



appeared, the student typed the fact. If the 
student made an error, the correct fact was 
displayed with accompanying audio, and 
the student had the opportunity to type 
the fact again. For two of the studies, the 
duration of the presentation on the screen 
was tailored to the student’s performance 
(with less time as the student gained profi- 
ciency) and the difficulty of facts increased 
as competency increased. ^69 

Time. The panel advocates dedicating 
about 10 minutes a session to building fact 
fluency in addition to the time dedicated to 
tier 2 and tier 3 interventions. The seven 
studies supporting this recommendation 
dedicated a minimum of 10 minutes a ses- 
sion to fact fluency activities. 

Explicit teaching strategies for building 
fact fluency. Another three studies in the 
evidence base address methods for teach- 
ing basic facts to students by comparing 
instructional approaches. Both Bernie- 
Smith (1991) and Tournaki (2003) investi- 
gated the effects of being taught a counting 
up/on strategy relative to a rote memoriza- 
tion procedure for promoting fact fluency. 
In the Bernie-Smith (1991) study, perhaps 
not surprisingly as both interventions 
were taught to enhance fact fluency, the 
addition facts competency of students in 
both groups improved. However, there 
was a substantively important nonsignifi- 
cant positive effect in favor of the count- 
ing-on group when the two groups were 
compared. In the Tournaki (2003) study, 
the latency of responses on a fact fluency 
posttest decreased^^^ while the accuracy of 
posttest responses significantly increased 



269. Fuchs et al. (2005); Fuchs, Fuchs, Hamlett 
et al. (2006). 

270. Bernie-Smith (1991); Tournaki (2003); Wood- 
ward (2006). 

271. The latency decrease was marginally sig- 
nificant. A decrease in latency indicates that stu- 
dents in the counting-on group were answering 
fact problems more quickly than students in the 
rote memorization group. 
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for the counting-on group relative to the 
rote memorization group. 

Similarly, Woodward (2006) examined 
an integrated approach that combined 
instruction in strategies, visual repre- 
sentations, and timed practice drills (the 
Integrated Strategies group) versus a tra- 
ditional timed drill and practice approach 
for building multiplication fact fluency (the 
Timed-Practice Only group). In the Inte- 
grated Strategies group, difficult facts were 
taught through derived fact strategies or 
doubling and doubling again strategies. 
When WWC multiple comparison adjust- 
ments were applied to outcomes, none 
of the multiplication fact outcomes were 
significant, though effects were substan- 
tively important and positive in favor of 
the integrated approach. operations 
domain showed mixed effects with approx- 
imation scores in favor of the integrated 
approach and operations scores in favor 
of the Timed-Practice Only group. 

In summary, the evidence demonstrates 
substantively important or statistically 
significant positive effects for including 
fact fluency activities as either stand-alone 
interventions or components of larger tier 
2 interventions. However, because these 
effects did not consistently reach statisti- 
cal significance, the panel is cautious and 
acknowledges that the level of evidence 
for this recommendation is moderate. 
There is also evidence that strategy-based 
instruction for fact fluency (e.g., teaching 
the counting-on procedure) is a superior 
approach over rote memorization. Further, 
many of the studies included here taught 
the relationships among facts, used a va- 
riety of materials such as flash cards and 
computer assisted instruction, and taught 
math facts for a minimum of 10 minutes a 



272. When a study examines many outcomes or 
findings simultaneously, the statistical signifi- 
cance of findings may be overstated. The WWC 
makes a multiple comparison adjustment to pre- 
vent drawing false conclusions about the number 
of statistically significant effects (WWC, 2008). 



session. Although these components of the 
interventions were most often not inde- 
pendent variables in the studies, they are 
all advocated by the panel. An example of 
a study investigating the effects of a fact 
fluency intervention is detailed here. 

A study of a fact fluency intervention— 
Fuchs, Powell, Hamlett, and Fuchs (2008). 

This study was conducted with 127 stu- 
dents in grade 3 classrooms in Tennessee 
and Texas. The students were all iden- 
tified as having either math difficulties or 
math and reading difficulties. 

Before the intervention, the students com- 
pleted several pretest assessments. The 
assessment that related to fact retrieval 
consisted of one subtest with three sets 
of 25 addition fact problems and a second 
subtest with three sets of 25 subtraction 
fact problems. Students had one minute to 
write answers for each set within a subtest. 
Internal consistency for the sets ranged be- 
tween .88 and .93. Scores on sets of items 
were combined into a single fact retrieval 
score. After the intervention, students 
completed this same fact retrieval assess- 
ment among a battery of posttests. 

Students were randomly assigned to one 
of four groups (three mathematics instruc- 
tion groups and a reading instruction com- 
parison group). For this recommendation, 
we report only on the comparison between 
the Fact Retrieval group (n = 32) and the 
Word Identification comparison group (n 
= 35). Students in both groups met in- 



273. This study met standards with reservations 
because of high attrition. The sample initially 
included 165 students randomized to the condi- 
tions and 127 in the postattrition sample. The 
authors did demonstrate baseline equivalence 
of the postattrition sample. 

274. The third group was Procedural/Estimation 
Tutoring, which targeted computation of two- 
digit numbers. The fourth group was a combi- 
nation of Procedural/Estimation Tutoring and 
Eact Retrieval. 
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dividually with a tutor^^^ for 15 to 18 min- 
utes during three sessions each week for 
15 weeks. 

Sessions for the Fact Retrieval instruction 
group consisted of three activities. First, 
the students received computer assisted 
instruction (CAI). In the computer pro- 
gram, an addition or subtraction math- 
ematics fact appeared on the screen for 
1.3 seconds. When the fact disappeared, 
the student typed the fact using short- 
term memory. A number line illustrated 
the mathematics fact on the screen with 
colored boxes as the student typed. If the 
student typed the fact correctly, applause 
was heard, and the student was awarded 
points. Each time the student accumulated 
five points, animated prizes (e.g., a pic- 
ture of a puppy) appeared in the student’s 
animated “treasure chest.” If the student 
typed the mathematics fact incorrectly, 
the fact reappeared and the student was 
prompted to type it again. 

The second instructional activity, flash 
card practice, began after 7.5 minutes of 
CAI. Flash card practice with corrective 
feedback included two types of flash cards. 
The first set of flash cards depicted writ- 
ten facts without answers. Students were 
encouraged to answer as many problems 



275. There were 22 tutors. Some were masters 
or doctoral students. Most had teaching or tutor- 
ing experience. 



as possible in two minutes. After three 
consecutive sessions with a minimum of 
35 correct responses, the student was pre- 
sented with a second set of flash cards that 
contained a number line similar to the CAI 
number line. The student was asked to re- 
spond with the appropriate mathematics 
facts to accompany the number line for 
as many cards as possible within the time 
frame. Corrective feedback was provided 
for a maximum of five errors per flash card 
activity. The third activity during Fact Re- 
trieval instruction focused on cumulative 
review. Students were allotted two minutes 
to complete 15 mathematics fact problems 
using paper and pencil. 

Students in the Word Identification com- 
parison group received computer assisted 
instruction and participated in repeated 
reading with corrective feedback during 
their sessions. The content was tailored 
to the student’s reading competency level 
as determined by a pretest. 

Results indicated significant positive ef- 
fects on fact fluency in favor of the group 
that received fact retrieval instruction 
relative to the comparison group that re- 
ceived instruction in word identification. 
These results suggest that it is possible 
to teach struggling students mathematics 
facts in as small an amount of time as 45 
minutes of instruction a week when using 
flash cards and CAI. 
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Recommendation 7. Monitor the 
progress of students receiving 
supplemental instruction and other 
students who are at risk. 

Level of evidence: Low 

The panel rated the level of evidence for 
this recommendation as low. The panel 
relied on the standards of the American 
Psychological Association, the American 
Educational Research Association, and the 
National Council on Measurement Educa- 
tion^^s for valid assessment instruments, 
along with expert judgment, to evaluate 
the quality of the individual studies and 
to determine the overall level of evidence 
for this recommendation. 

Evidence for the recommendation included 
research studies on mathematics progress 
monitoring,277 summary reviews of math- 
ematics progress monitoring research, 
and summary information provided by the 
Research Institute on Progress Monitor- 
ing279 and the National Center on Progress 
Monitoring.280 Very little research evidence 
specifically addresses the use of math- 
ematics progress monitoring data within 
the context of Rtl. 

Most research on mathematics progress 
monitoring measures falls into two cat- 
egories. One group of studies examines 
the technical adequacy of the measures, 
including their reliability, validity, and 
sensitivity to growth. The second investi- 
gates teacher use of the measures to mod- 
ify instruction for individual students in 
order to enhance achievement; the bulk of 



276. American Educational Research Association, 
American Psychological Association, and National 
Council on Measurement in Education (1999). 

277. Clarke et al. (2008); Eoegen and Deno (2001); 
Euchs et al. (1993); Euchs, Euchs, Hamlett, Thompson 
et al. (1994); Eeh et al. (2007); Eembke et al. (2008). 

278. Eoegen, Jiban, and Deno (2007). 

279. www.progressmonitorlng.net/. 

280. www.studentprogress.org/. 



this second body of research has been con- 
ducted primarily in special education set- 
tings and therefore is less relevant to the 
Rtl focus of this practice guide. As a result, 
we focus on the technical adequacy studies 
in this appendix. Note that because similar 
and often identical measures are used for 
screening and progress monitoring, many 
of the studies reviewed here overlap with 
those discussed for recommendation 1 
on screening. The same measure may be 
used as both a screening measure and a 
progress monitoring measure; however, 
the psychometric properties of these mea- 
sures are more firmly established when 
used as screening measures with fewer 
researchers investigating the function of 
the measures for modeling growth when 
used for progress monitoring. This dispar- 
ity in the research base leads to the panel 
assigning a moderate level of evidence to 
Recommendation 1 and a low level of evi- 
dence to Recommendation 7. 

The technical adequacy studies of math- 
ematics progress monitoring measures 
were not experimental; the researchers 
typically used correlational techniques to 
evaluate the reliability and criterion valid- 
ity of the measures and regression meth- 
ods to examine sensitivity to growth. If 
progress monitoring measures are to be 
deemed trustworthy, relevant empiri- 
cal evidence includes data on reliability, 
concurrent criterion validity, and sensi- 
tivity to growth. Evidence of reliability 
generally includes data on inter-scorer 
agreement, internal consistency,^®^ test- 
retest reliability,^®® and alternate form 
reliability.®®'^ Evidence of concurrent cri- 
terion validity is gathered by examining 
relations between scores on the progress 
monitoring measures and other indica- 



281. Eor example, Euchs, Euchs, Hamlett, Thomp- 
son et al. (1994). 

282. Eor example, Jitendra, Sczesniak, and Deat- 
line-Buchman (2005). 

283. Eor example, Clarke and Shinn (2004). 

284. VanDerHeyden et al. (2001). 
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tors of proficiency in mathematics. Com- 
mon criterion measures include scores 
on group and individual standardized 
tests of mathematics, course grades, and 
teacher ratings. 

Although issues of reliability and criterion 
validity are common to both screening and 
progress monitoring measures, a critical 
feature specific to progress monitoring is 
that the measures be sensitive to growth. 
If teachers are to use progress monitor- 
ing measures to evaluate the effects of in- 
struction on student learning, researchers 
must provide evidence that student scores 
on the measures change over time, thus 
providing an indication of their learning. 
Most research studies have examined sen- 
sitivity to growth by administering paral- 
lel forms of a measure across a period of 
several weeks or months. In some studies, 
students receive typical instruction, and 
in others, teachers adapt and refine the 
instruction in response to the progress 
monitoring data (often in special educa- 
tion contexts). In either case, evidence of 
sensitivity to growth typically involves 
computing regression equations to deter- 
mine slopes of improvement and report- 
ing these as mean weekly growth rates 
for a group of students. As an example, 
if a progress monitoring measure has a 
mean weekly growth rate of .5, teachers 
could expect that, on average, a student’s 
score would increase by 1 point every two 
weeks. Growth rates reported in the litera- 
ture vary considerably across measures 
and grade levels; no established stan- 
dards exist for acceptable rates of student 
growth under typical instruction. 

We discuss the evidence for measures 
used across the elementary and middle 
school grades and conclude with a more 
in-depth example of a technical adequacy 
study of mathematics progress monitor- 
ing measures. 



285. For example, Foegen and Deno (2001); Fuchs 
et al. (2003a); Chard et al. (2005). 



Summary of evidence 

Progress monitoring in the primary grades. 
Measures for the primary grades typically 
reflect aspects of number sense, including 
strategic counting, numeral identification, 
and magnitude comparison. Among the 
studies examining sensitivity to growth 
in the early grades, researchers have re- 
lied on separate measures for each of the 
different aspects of numeracy.^®® Other 
researchers have combined individual 
measures to create composite scores^®^ 
or used more comprehensive multiskill 
measures. But so far, the focus of these 
studies has been on screening rather than 
on progress monitoring. Reliability coef- 
ficients for these measures generally ex- 
ceed .85. Concurrent criterion validity co- 
efficients with standardized achievement 
tests are generally in the .5 to .7 range. 
Mean rates of weekly growth reported in 
the literature vary widely, ranging from .1 
to .3^®° problems a week to .2 to more than 
1.0 problems. 

Progress monitoring in the elementary 
grades. Two types of measures have been 
investigated for monitoring the mathemat- 
ics learning of students in the elementary 
grades. The bulk of the research, con- 
ducted by a research group led by Dr. Lynn 
Fuchs, investigates the characteristics of 
general outcome measures that represent 
grade-level mathematics curricula in com- 
putation and in mathematics concepts and 
applications. These measures were de- 
veloped in the late 1980s and early 1990s, 
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reflecting the Tennessee state elementary 
mathematics curriculum of that time. The 
measures continue to be widely used and 
are recommended by the National Center 
for Student Progress Monitoring. Teach- 
ers should carefully examine the content 
of the measures to ensure that they are 
representative of the existing mathematics 
curricula in their states and districts. 

The second type of measure is not broadly 
representative of the instructional curricu- 
lum as a whole, but instead serves as an 
indicator of general proficiency in math- 
ematics. Examples of such measures in- 
clude basic facts (number combinations)^®'^ 
and word problem solving. Because the 
general outcome measures are representa- 
tive of the broader curriculum, they offer 
teachers more diagnostic information 
about student performance in multiple 
aspects of mathematics competence; this 
advantage is often gained by using mea- 
sures that are longer and require more ad- 
ministration time. The indicator measures 
are more efficient to administer for regular 
progress monitoring but may be as useful 
for diagnostic purposes. 

Evidence of the reliability of the measures 
is generally strong, with correlation co- 
efficients above .8, except for the word 
problem-solving measures developed by 
Jitendra’s research team, which are slightly 
lower.2®® Concurrent criterion validity mea- 
sures have included group and individual 
achievement tests. Validity correlation coef- 
ficients range widely across measure types 
and grade levels. At the lower end, Espin et 
al. (1989) found correlations between the 
Wide Range Achievement Test and basic 
fact measures in the .3 to .5 range for basic 
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facts. In contrast, Euchs, Hamlett, and Euchs 
(1998) found correlations between the Stan- 
ford Achievement Test Math Computation 
subtest^®^ and general outcome measures 
of computation to range from .5 to .9 across 
grades 2 through 5. In general, concurrent 
criterion validity coefficients for elementary 
mathematics progress monitoring measures 
are in the .5 to .6 range. 

Evidence of sensitivity to growth for el- 
ementary measures exists for the com- 
putation and concepts/applications mea- 
sures developed by Euchs and for the word 
problem-solving measures developed by 
Jitendra. Mean growth rates for the Euchs 
measures range from .25 to .70. A study by 
Shapiro and colleagues, ^®® using the same 
measures for students with disabilities, re- 
sulted in mean growth rates of .38 points 
per week, for both types of measures. Mean 
weekly growth rates for the Jitendra mea- 
sures were .24 points per week. 

Progress monitoring in middle school. Less 
evidence is available to support progress 
monitoring in middle school. ^99 Research 
teams have developed measures focus- 
ing on math concepts typically taught in 
middle school,^®® basic facts®®^ and esti- 
mation. ®®^ Criterion validity across the 
types of measures varies, but the majority 
of correlations coefficients fall in the .4 to 
.5 range. Helwig and colleagues^®^ found 
higher correlation coefficients with high- 
stakes state tests in the range of .6 to .8. 
Reliability estimates including alternate 
form, inter-rater, and test-retest were all of 
sufficient quality. Greater rates of growth 
were found for the non-concept-based 
measures with rates around .25 units per 
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week. A recent study^°^ compared these 
measure types with two grade 6 mea- 
sures^^^ similar to the measures described 
above assessing student understanding of 
operations and concepts for their grade 
level. In this case, middle school students 
in grades 6, 7, and 8 were assessed using 
multiple measures. Evidence was found 
that even into grades 7 and 8, using grade 
6 measures focusing on operations and 
mathematical concepts still shows reliabil- 
ity, validity, and sensitivity to growth. 

An example of a study of the technical 
adequacy of mathematics progress 
monitoring measures — Fuchs, Fuchs, 
Hamlett, Thompson, Roberts, Kupek, 
and Stacker (1994) 

A study conducted by Fuchs, Fuchs, Ham- 
lett, Thompson, Roberts, Kupek, and 
Stacker (1994) illustrates the type of tech- 
nical adequacy evidence that education 
professionals can use when evaluating and 
selecting mathematics progress monitor- 
ing measures. The research team exam- 
ined the technical features of grade-level 
general outcome measures of mathematics 
concepts and applications. The measures 
were developed by analyzing the Tennes- 
see mathematics curriculum at grades 2 
through 6 to identify critical objectives 
essential for mathematics proficiency at 
each grade level. The researchers created 
30 alternate forms at each grade level and 
conducted numerous pilot tests to refine 
the items and determine appropriate time 
limits for administration. 

A total of 140 students in grades 2 through 4 
participated in the study, completing weekly 
versions of the measures for 20 weeks. All 
students were enrolled in general education 
classrooms; about 8 percent of the students 
had been identified as having learning dis- 
abilities. The students’ general education 
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teachers administered the measures using 
standardized procedures, including admin- 
istration time limits of 6 to 8 minutes, de- 
pending on grade level. 

The results of the study illustrate the types 
of data educators should consider to deter- 
mine if a mathematics progress monitor- 
ing measure is trustworthy. The authors 
report evidence of the reliability of the 
concepts and application measures by de- 
scribing internal consistency coefficients 
for students at each grade level (which 
ranged from .94 to .98). Concurrent crite- 
rion validity was examined by computing 
correlations between student scores on 
the concepts and applications general out- 
come measures and their scores on three 
subscales of the Comprehensive Test of 
Basic Skills (Computation, Concepts and 
Applications, and Total Math Battery). 
Results are reported for each subscale at 
each of the three grade levels, with coef- 
ficients ranging from .63 to .81. Consider- 
ing these results in the general context of 
mathematics progress monitoring mea- 
sures summarized above, teachers could 
feel confident that the concepts and ap- 
plications measures demonstrated strong 
levels of reliability and criterion validity 
in this study. 

A final consideration is the degree to which 
the measures are sensitive to student 
growth. To explore this feature, the re- 
searchers completed a least-squares re- 
gression analysis between calendar days 
and scores on the progress monitoring 
measures; the scores were then converted 
to represent weekly rates of improvement. 
The results ranged from an average in- 
crease of .40 problems per week in grade 
2 to .69 in grade 4. Together with the evi- 
dence of reliability and criterion validity, 
the mean growth rate data suggest that 
teachers can have confidence that students 
will show improvements in their scores on 
the measures as their mathematics learn- 
ing progresses. 
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One factor not evident in the technical ad- 
equacy data in this study, but critical for 
teachers to consider, is the alignment be- 
tween the general outcome measure, the 
instructional curriculum, and expected 
learning outcomes. This study produced 
strong technical adequacy data when it 
was conducted in the early 1990s. Teach- 
ers considering alternative mathematics 
progress monitoring measures to rep- 
resent the instructional curriculum are 
advised to review the content of these 
measures in light of current learning ex- 
pectations for students at each grade level. 
Given changes in mathematics curricula 
over the past 10 to 15 years, it is impor- 
tant to evaluate the degree to which the 
measures continue to represent important 
mathematics outcomes. 

Recommendation 8. Include 
motivational strategies in tier 2 
and tier 3 interventions. 

Level of evidence: Low 

The panel judged the level of evidence sup- 
porting this recommendation to be low. The 
panel found nine studies^°® conducted with 
low-achieving or learning disabled students 
between grades 1 and 8 that met WWC stan- 
dards or met standards with reservations 
and included motivational strategies in an 
intervention. However, because only two of 
these studies investigated a motivational 
strategy in a tier 2 or tier 3 mathematics in- 
tervention as an independent variable, the 
panel concluded that there is /ow evidence 
to support the recommendation. The panel 
recommends this practice for students in 
tier 2 and tier 3 based both on our opinion 
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and on the limited evidence base.^°^ The 
evidence base is described below. 

Summary of evidence 

Reinforce effort. The panel advocates re- 
inforcing or praising students for their ef 
fort. Schunk and Cox (1986) examined the 
effects of providing effort-attributional 
feedback (e.g., “You’ve been working hard”) 
during subtraction instruction versus no 
effort feedback and found significant posi- 
tive effects on subtraction posttests in 
favor of providing effort feedback. This 
study, described in greater detail below, 
was one of two studies in the evidence 
base that examined a motivational strat- 
egy as an independent variable. 

Reinforce engagement. The panel also rec- 
ommends reinforcing students for attending 
to and being engaged in lessons. In two of 
the studies, students received “points” for 
engagement and attentiveness as well as for 
accuracy.^°® Accumulated points could be 
applied toward “purchasing” tangible rein- 
forcers. It is not possible to isolate the effects 
of reinforcing attentiveness in the studies. 
In Fuchs et al. (2005), both the treatment 
and comparison groups received reinforce- 
ment, and in Fuchs, Fuchs, Craddock et al. 
(2008), the contrast between reinforcement 
and no reinforcement was not reported. But 
the presence of reinforcers for attention and 
engagement in these two studies echoes the 
panel’s contention that providing reinforce- 
ment for attention is particularly important 
for students who are struggling. 



307. The scope of this practice guide limited the 
evidence base for this recommendation to stud- 
ies that investigated mathematics interventions 
for students with mathematics difficulties and 
included motivational components. There is an 
extensive literature on motivational strategies 
outside the scope of this practice guide, and the 
panel acknowledges that there is considerable 
debate in that literature on the use of rewards 
as reinforcers. 

308. Fuchs et al. (2005); Fuchs, Fuchs, Craddock 
et al. (2008). 



88 



APPENDIX D. TECHNICAL INFORMATION ON THE STUDIES 



Consider rewarding accomplishments. The 
panel recommends that interventionists 
consider using rewards to acknowledge 
accurate work and possibly notifying par- 
ents when students demonstrate gains. In 
three of the studies, students were pro- 
vided prizes as tangible reinforcers for ac- 
curate mathematics problem solving.^°® In 
both Fuchs et al. (2005) and Fuchs, Seetha- 
ler et al. (2008), students in tier 2 tutoring 
earned prizes for accuracy. In both stud- 
ies, the tier 2 intervention group demon- 
strated substantively important positive 
and sometimes significant gains relative to 
the students who remained in tier 1. But it 
is not possible to isolate the effects of the 
reinforcers from the provision of tier 2 
tutoring. In Fuchs, Fuchs, Craddock et al. 
(2008), the authors note that the provision 
of “dollars” that could be exchanged for 
prizes was more effective than rewarding 
students with stickers alone. Because this 
was not the primary purpose of the study, 
the reporting of the evidence for that find- 
ing was not complete; therefore, a WWC re- 
view was not conducted for that finding. 

In a fourth study, Heller and Fantuzzo 
(1993) examined the impacts of a parental 
involvement supplement to a mathematics 
intervention. The parental involvement 
component included parents providing 
rewards for student success as well as pa- 
rental involvement in the classroom. The 
performance of students who received 
the parental involvement component in 
addition to the school-based intervention 
significantly exceeded the performance of 
students in only the school-based inter- 
vention. Because the parental involvement 
component was multifaceted, it is not pos- 
sible to attribute the statistically signifi- 
cant positive effects to rewards alone. 

Allow students to chart their progress and 
to set goals for improvement. Five studies 
included interventions in which students 
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graphed their progress and in some cases 
set goals for improvement on future as- 
sessments.^^® One experimental study 
examined the effects of student graphing 
and goal setting as an independent vari- 
able and found substantively important 
positive nonsignificant effects in favor of 
students who graphed and set goals. In 
two studies, the interventions included 
graphing in both groups being compared; 
therefore, it was not possible to isolate the 
effects of this practice. In another two 
studies, students in the treatment groups 
graphed their progress as one component 
of multicomponent interventions. Al- 
though it is not possible to discern the ef- 
fect of graphing alone, in Artus and Dyrek 
(1989), the treatment group made margin- 
ally significant gains over the comparison 
group on a general mathematics assess- 
ment, and in Fuchs, Seethaler et al. (2008), 
there were substantively important posi- 
tive non significant effects on fact retrieval 
in favor of the treatment group. 

In summary, the evidence base for motiva- 
tional components in studies of students 
struggling with mathematics is limited. 
One study that met evidence standards 
demonstrated benefits for praising strug- 
gling students for their effort. Other stud- 
ies included reinforcement for attention, 
engagement, and accuracy. Because the ef- 
fects of these practices were not examined 
as independent variables, no inferences 
can be drawn about effectiveness based on 
these studies. Because this recommenda- 
tion is based primarily on the opinion of 
the panel, the level of evidence is identi- 
fied as low. 
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An example of a study that investigated 
a motivational component— Schunk 
and Cox (1986). 

This study was conducted with 90 stu- 
dents in grades 6 through 8 classrooms 
in six schools in Texas. The mean age of 
the students was 13 years and 7 months, 
and all students were identified as having 
learning disabilities in mathematics. 

Before the intervention, the students com- 
pleted a pretest assessment that consisted 
of 25 subtraction problems that required 
regrouping operations. After the interven- 
tion, a similar assessment of 25 subtrac- 
tion problems was completed as a post- 
test. A separate reliability assessment 
demonstrated that the two forms of the 
subtraction assessment were highly cor- 
related (r= .82). 

Students were stratified by gender and 
school and then randomly assigned to one 
of nine experimental groups. In all groups, 
the students received instruction for solv- 
ing subtraction problems in 45-minute 
sessions conducted over six consecutive 
school days. For this recommendation. 



we report only on the comparison be- 
tween three groups. One group (n = 30) 
received effort feedback in addition to per- 
formance feedback during the first three 
sessions. Another group (n = 30) received 
effort feedback in addition to performance 
feedback during the last three sessions. 
A third group (n = 30) did not receive ef- 
fort feedback (received only performance 
feedback). Effort feedback consisted of 
the proctor commenting to the student, 
“You’ve been working hard.” Students in 
both effort feedback groups received 15 
statements of effort feedback across the 
entire intervention. 

Results indicated significant positive ef- 
fects for effort feedback relative to the 
comparison group regardless of when the 
student received the effort feedback. These 
results suggest that effort feedback is ben- 
eficial for learning disabled students who 
may not otherwise recognize the causal 
link between effort and outcomes. 



314. Other group distinctions were related 
to student verbalization and are described in 
the discussion of recommendation 3 (explicit 
instruction). 
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